Systems and methods for diagnosing tumors in a subject by performing a quantitative analysis of texture-based features of a tumor object in a radiological image

ABSTRACT

An example method for diagnosing tumors in a subject by performing a quantitative analysis of a radiological image can include identifying a region of interest (ROI) in the radiological image, segmenting the ROI from the radiological image, identifying a tumor object in the segmented ROI and segmenting the tumor object from the segmented ROI. The method can also include extracting a plurality of quantitative features describing the segmented tumor object, and classifying the tumor object based on the extracted quantitative features. The quantitative features can include one or more texture-based features.

STATEMENT REGARDING FEDERALLY FUNDED RESEARCH

This invention was made with Government support under Grant No. CA143062awarded by the National Institutes of Health. The Government has certainrights in the invention.

BACKGROUND

Lung cancer is responsible for the greatest number of cancer relateddeaths in the United States. The primary precursor to lung cancer is thedevelopment of pulmonary nodules. Several diagnostic procedures areavailable to detect these nodules including pulmonary function tests,blood tests, biopsy, and imaging tests such as X-ray, magnetic resonanceimaging (MRI), computed tomography (CT), and positron emissiontomography (PET). Regardless of the diagnostic procedure used, theultimate diagnosis is usually confirmed by completing a biopsy.

The advent of low-dose helical computed tomography has made it possibleto provide relatively low risk screening for high risk patients. Thoughstill somewhat controversial, results of the National Lung ScreeningTrial (NLST) have shown a 20% decrease in mortality with the use oflow-dose CT compared to X-ray findings. The sensitivity of the procedureis its bane; many of the detected nodules are not cancerous.

Accurate classification (or prediction) of pulmonary nodules to becancerous is key to determining further diagnosis and treatment options.In order to provide medical personnel with the information to accuratelydetermine the nature of the nodule, several computer aided diagnosissystems have been created to classify pulmonary nodules. Traditionally,these systems have been based on size and volume measurements.

SUMMARY

An example method for diagnosing tumors in a subject by performing aquantitative analysis of a radiological image can include identifying aregion of interest (ROI) in the radiological image, segmenting the ROIfrom the radiological image, identifying a tumor object in the segmentedROI and segmenting the tumor object from the segmented ROI. The methodcan also include extracting a plurality of quantitative featuresdescribing the segmented tumor object, and classifying the tumor objectbased on the extracted quantitative features. The quantitative featurescan include one or more texture-based features with or without size andshape features.

Optionally, predicting/classifying the indeterminate pulmonary nodule tobecome tumor/malignant or benign may be based on the extractedquantitative features. Further, the tumor object may be either a noduleor a mass that comprises of all anomalies, including ground-glassopacity (GGO) nodules. Predicting/classifying patient tumor (lesions)prognosis /relapse or progression may be based on extracted quantitativefeatures. The tumor object may include all types of abnormalitiesincluding, but not limited to, ground glass opacities (GGO), histologyand staging. The prediction/classification could be for a sunset ofhistology and staging (based on TNM scale). Alternatively oradditionally, the tumor object can be classified using a decision treealgorithm, a nearest neighbor algorithm or a support vector machine. Itshould be understood that these classification algorithms are providedonly as examples and that other known classification algorithms can beused.

Alternatively or additionally, each of the texture-based featuresdescribes a spatial arrangement of image intensities within the tumorobject. For example, the texture-based features can include a run-lengthtexture feature, a co-occurrence texture feature, a Laws texturefeature, a wavelet texture feature, gray level (Hounsfield scale)histogram texture feature. Additionally, the quantitative features canoptionally include one or more shape-based features. Each of theshape-based features can describe a location, quantification of extendof attachment to the lung wall (circumference of tumor to attachmentextent), a geometric shape, a volume, a surface area, asurface-area-to-volume ratio or a compactness of the tumor object.

The ROI can optionally be a lung field. It should be understood that alung field is provided only as an example. This disclosure contemplatesthat the ROI can be a region in another organ of the subject.Optionally, the radiological image is a low-dose computed tomography(CT) image. Although a low-dose CT image is provided as an example, thisdisclosure contemplates that the radiological image can be another typeof image, including but not limited to, an X-ray image, an MRI and a PETimage.

Optionally, the method can further include reducing the extractedquantitative features to a subset of extracted quantitative features,and classifying the tumor object based on the subset of extractedquantitative features. For example, the subset of extracted quantitativefeatures can optionally include one or more quantitative features thatare predictive of being a tumor. Accordingly, the method can optionallyinclude determining the one or more quantitative features that arepredictive of the classification of the tumor object using at least oneof a Recursive Elimination of Features (Relief-F) algorithm, aCorrelation-Based Feature Subset Selection for Machine Learning (CFS)algorithm or a Relief-F with Correlation Detection algorithm oriterative search.

Alternatively or additionally, the subset of extracted quantitativefeatures can optionally include one or more non-redundant quantitativefeatures having adequate reproducibility and dynamic range. For example,to reduce the extracted quantitative features to a subset of extractedquantitative features, the method can optionally include eliminating oneor more of the extracted quantitative features which are not part ofreproducible features based on test retest repeatable study wherebaseline radiological image and a subsequent radiological image lessthan a predetermined reproducibility value. The subsequent radiologicalimage is an image captured a fixed period of time after the baselineradiological image was captured. The reproducibility metric is aconcordance correlation coefficient (CCC) or Interclass correlationcoefficient (ICC). This disclosure contemplates using other knownmetrics for the reproducibility of a feature between baseline andsubsequent images. Optionally, the predetermined reproducibility valueis less than 0.90. Although 0.90 is provided as an example of thepredetermined reproducibility value, it should be understood that valuesmore or less than 0.90 can be used.

Alternatively or additionally, to reduce the extracted quantitativefeatures to a subset of extracted quantitative features, the method canoptionally include eliminating one or more of the extracted quantitativefeatures having a dynamic range less than a predetermined dynamic rangevalue. Optionally, the predetermined dynamic range value is greater than0.55 (e.g., 55% of its dynamic range). Although 0.55 is provided as anexample of the predetermined dynamic range value, it should beunderstood that values more or less than 0.55 can be used.

Alternatively or additionally, to reduce the extracted quantitativefeatures to a subset of extracted quantitative features, the method canoptionally include eliminating one or more of the extracted quantitativefeatures that are redundant quantitative features. For example, themethod can include calculating a coefficient of determination (R²_(Bet)) between a pair of (least two) quantitative features.Accordingly, one or more quantitative features having an R² _(Bet)greater than a predetermined redundancy value can be eliminated Itshould be understood that these eliminated quantitative features arehighly correlated and are considered redundant quantitative features.Optionally, the predetermined redundancy value is greater than 0.95.Although 0.95 is provided as an example of the predetermined redundancyvalue, it should be understood that values more or less from 0.75 to0.99 are relevant, appropriate levels can be used.

It should be understood that the above-described subject matter may alsobe implemented as a computer-controlled apparatus, a computer process, acomputing system, or an article of manufacture, such as acomputer-readable storage medium.

Other systems, methods, features and/or advantages will be or may becomeapparent to one with skill in the art upon examination of the followingdrawings and detailed description. It is intended that all suchadditional systems, methods, features and/or advantages be includedwithin this description and be protected by the accompanying claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The components in the drawings are not necessarily to scale relative toeach other. Like reference numerals designate corresponding partsthroughout the several views.

FIG. 1 is a flow diagram illustrating example operations for diagnosingtumors in a subject by performing a quantitative analysis of aradiological image as described herein;

FIG. 2 is an example segmented ROI in a radiological image;

FIG. 3 is an example segmented object in a radiological image;

FIG. 4 is a block diagram of an example computing device;

FIG. 5 illustrates examples of CT scans for extreme semantic scores;

FIG. 6 illustrates a heatmap of coefficient of determination betweenfeatures;

FIGS. 7A and 7B illustrate a receiver operator characteristics (ROC) fortwo features;

FIGS. 8A and 8B illustrate two sample cases with representative slicesfor Run length and a Laws kernel; and

FIG. 9 illustrates the relationship of prognostic texture features toconventional measures.

DETAILED DESCRIPTION

Unless defined otherwise, all technical and scientific terms used hereinhave the same meaning as commonly understood by one of ordinary skill inthe art. Methods and materials similar or equivalent to those describedherein can be used in the practice or testing of the present disclosure.As used in the specification, and in the appended claims, the singularforms “a,” “an,” “the” include plural referents unless the contextclearly dictates otherwise. The term “comprising” and variations thereofas used herein is used synonymously with the term “including” andvariations thereof and are open, non-limiting terms. The terms“optional” or “optionally” used herein mean that the subsequentlydescribed feature, event or circumstance may or may not occur, and thatthe description includes instances where said feature, event orcircumstance occurs and instances where it does not. Whileimplementations will be described for diagnosing tumors in a subject byperforming a quantitative analysis of a low-dose CT image, it willbecome evident to those skilled in the art that the implementations arenot limited thereto, but are applicable for diagnosing tumors in asubject by performing a quantitative analysis of other types ofradiological images.

Low-dose helical computed tomography (LDCT) has facilitated the earlydetection of lung cancer through pulmonary screening of patients. Therehave been a few attempts to develop a computer-aided diagnosis systemfor classifying pulmonary nodules using size and shape, with littleattention to texture features. As described herein, texture and shapefeatures were extracted from pulmonary nodules selected from the LungImage Database Consortium (LIDC) data set. Several classifiers includingDecision Trees, Nearest Neighbor, linear discriminant and Support VectorMachines (SVM) were used for classifying malignant and benign pulmonarynodules. An accuracy of 90.91% was achieved using a 5-nearest-neighborsalgorithm and a data set containing texture features only. Laws andWavelet features received the highest rank when using feature selectionimplying a larger contribution in the classification process.Considering the improvement in classification accuracy, it is possibleto perform computer-aided diagnosis of pulmonary nodules in LDCT byperforming a quantitative analysis of texture features.

Referring now to FIG. 1, a flow diagram illustrating example operationsfor diagnosing tumors in a subject by performing a quantitative analysisof a radiological image is shown. In the examples below, theradiological image is a low-dose computed tomography (CT) image.Although a low-dose CT image is provided as an example, this disclosurecontemplates that the radiological image can be another type of image,including but not limited to, an X-ray image, an MRI and a PET image. At102, a region of interest (ROI) in the radiological image is identifiedand then segmented from the radiological image. An example segmented ROI202 in a radiological image is shown in FIG. 2. In particular, thesegmented ROI 202 is a segmented lung field in FIG. 2. It should beunderstood that the ROI can be a region in another organ of the subjectand that a lung field is provided only as an example. At 104, a tumorobject is identified in the segmented ROI and then segmented from theradiological image. An example segmented tumor object 302 in aradiological image is shown in FIG. 3. In particular, the segmentedtumor object 302 is a segmented pulmonary nodule in FIG. 3. Segmentingis a well-known technique in image processing, and this disclosurecontemplates using any manual, semi-automatic and/or automatic algorithmto segment the ROI and/or the tumor object from the radiological image.For example, the ROI and/or the tumor object can be segmented using aregion growing algorithm.

At 108, a plurality of quantitative features describing the segmentedtumor object can be extracted. The quantitative features can include oneor more texture-based features. As used herein, the texture-basedfeatures describe a spatial arrangement of image intensities within thetumor object. For example, the texture-based features can include arun-length texture feature, a co-occurrence texture feature, a Lawstexture feature, a wavelet texture feature or a histogram texturefeature. Each of these texture-based features is described in furtherdetail below. Additionally, Tables A1 and A2 below provide a descriptionof various quantitative features used to describe tumor objects. In someimplementations, a total number of quantitative features is optionallygreater than approximately 200. Additionally, a number of texture-basedfeatures is optionally greater than approximately 150. Although 219quantitative features are included in Table A2, it should be understoodthat more or less than 219 quantitative features can be extracted fromthe tumor object. In addition, about 119 features were obtained on asingle central slice referred as 2D features. The quantitative featuresprovided in Table A2 are provided as examples only (for 3D).

Run-length texture features examine runs of similar gray values in animage. Runs may be labeled according to their length, gray value, anddirection (either horizontal or vertical). Long runs of the same grayvalue correspond to coarser textures, whereas shorter runs correspond tofiner textures. Texture information can be quantified by computing 11features derived from the run-length distribution matrix. These 11feature are: 1: Short Run Emphasis (SRE). 2: Long Run Emphasis (LRE). 3:Gray-Level Non-uniformity (GLN). 4: Run Length Non-uniformity (RLN). 5:Run Percentage (RP). 6: Low Gray-Level Run Emphasis (LGRE). 7: HighGray-Level Run Emphasis (HGRE). 8: Short Run Low Gray-Level Emphasis(SRLGE). 9: Short Run High Gray-Level Emphasis (SRHGE). 10: Long Run LowGray-Level Emphasis (LRLGE). 11: Long Run High Gray-Level Emphasis(LGHGE).

Co-occurrence texture features are obtained from the co-occurrencematrix, which is a matrix that contains the frequency of one gray levelintensity appearing in a specified spatial linear relationship withanother gray level intensity within a certain range. Computation offeatures requires first constructing the co-occurrence matrix, thendifferent measurements can be calculated based on the matrix. Thedifferent measurements include: contrast, energy, homogeneity, entropy,mean and max probability.

Run-length & Co-occurrence features may find some correlation toradiologist visualized texture. The run length is defined as a measureof contiguous gray levels along a specific orientation. Fine texturestend to have short run length while coarser texture will have longer runlengths with similar gray level. These features capture coarseness in 3Dimage structure and have been found useful in a number of textureanalyses [20,21]. If R (k,p) is the run length matrix n₁ by n₂, at graylevel k then the number of such lengths equals p, along an orientation,in the volume (x,y,z). One useful measure of run length in this studyhas been the measure of Non-uniformity (RunL_(GLN)) which measuresextent of smoothness or similarity in the image.

${RunL}_{GLN} = \left( {{1/n}{\sum\limits_{k = 1}^{n_{1}}\; \left( {\sum\limits_{p = 1}^{n_{2}}\; {R\left( {k,p} \right)}} \right)^{2}}} \right)$

The co-occurrence matrices and run-length features can be obtained in3D, the features are calculated in 13 different directions, with eachdirection, processing is done by plane instead of slice. Hence,information between slices is not ignored.

Laws features were constructed from a set of five one-dimensionalfilters, each designed to reflect a different type of structure in theimage. These one-dimensional filters are defined as E5 (edges), S5(spots), R5 (ripples), W5 (waves), and L5 (low pass, or average grayvalue). By using these 1-D convolution filters, 2-D filters aregenerated by convolving pairs of these filters, such as L5L5, E5L5,S5L5, W5L5, R5L5, etc. In total, it is possible to generate 25 different2-D filters. 3D laws filters were constructed similarly to 2D. 3Dfilters are generated by convolving 3 types of 1D filter, such asL5L5L5, L5L5E5, L5L5S5, L5L5R5, L5L5W5, etc. The total number of 3-Dfilters is 125. For the 3D case, after the convolution with the 3Dfilters for the image, the energy of the texture feature can be computedby the following equation:

${{Energy} = {\frac{1}{R}{\sum\limits_{i = {N + 1}}^{I - N}\; {\sum\limits_{j = {N + 1}}^{J - N}\; {\sum\limits_{k = {N + 1}}^{K - N}\; {h^{2}\left( {i,j,k} \right)}}}}}},$

where R is a normalizing factor, I and J, K are image dimensions,h(i,j,k) is derived from the convolution filters and original image. Forthe 2D case, the above equation is very similar, but without the 3rd(e.g., z-direction) dimension.

Wavelet texture features are obtained using the discrete wavelettransform, which can iteratively decompose an image (2D) into fourcomponents. Each iteration involves convolving the image with waveletkernel both horizontally and vertically, followed by down sampling toobtain low-frequency (low pass) and high-frequency (high pass)components. Thus, doing it in both the directions four components aregenerated: a high-pass/high-pass component consisting of mostly diagonalstructure, a high-pass/low-pass component consisting mostly of verticalstructures, a low-pass/high-pass component consisting mostly ofhorizontal structure, and a low-pass/low-pass component that representsa blurred version of the original image. Subsequent iterations thenrepeat the decomposition on the low-pass/low-pass component from theprevious iteration. These subsequent iterations highlight broaderdiagonal, vertical, and horizontal textures. And for each component, theenergy (referred to with a suffix P1) & entropy (referred to with asuffix P2) of a feature is calculated. A wavelet transform of a 3Dsignal can be achieved by applying the 1D wavelet transform along allthe three directions (x,y,z). Featured obtained in each level ofdecomposition is referred with suffix L (example: L1, L2) and level ofdecomposition is referred to with a prefix C (example: C1 to C9).

Histogram texture features are obtained using the intensity histogram,h(a), which is the number of pixels that occurred for brightness level“a” plotted against their brightness level (in CT based, it isHounsfield units). The probability distribution of the brightness, P(a),can be calculated as well. Six features (e.g., mean, standard deviation,skewness, kurtosis, energy, and entropy) were then incorporated.

Alternatively or additionally, the quantitative features can optionallyinclude one or more shape-based features. In other words, thequantitative features can include a combination of texture-based andshape-based features. The shape-based features can describe a location,a geometric shape, a volume, a surface area, a surface-area-to-volumeratio or a compactness of the tumor object. Example shape-based featuresare also provided in Table A2.

TABLE A1 Quantitative Feature Categories Number of Category DescriptionDescriptors C1: Tumor Size Size, volume descriptors 13 C2: Tumor ShapeRoundness/circularity 12 (Roundness) descriptors C3: Tumor LocationRelative to pleural wall, 14 boarder flags C4: Pixel AttenuationStatistics on the Intensity 8 Histogram values (in HU) C5: Grayscale:Runlength Run length and Co-occurrence 17 & CoOccurrence patterns C6:Texture: Laws features Laws Kernel (energy) 125 C7: Texture: WaveletsWavelet kernels (entropy 30 and energy) Total 219

TABLE A2 Description of Quantitative Feature Used to Describe TumorObjects (e.g., a lung tumor) Sno Feature Index Description of theFeatures Feature Category 1 F1 Long Diameter C1.Tumor Size 2 F2 ShortAxis-LongDiameter 3 F3 Short Axis 4 F6 Volume (cm) 5 F33 Area-Pixels 6F34 Volume-pixels 7 F35 Num of Pixels 8 F36 Width-Pixels 9 F37Thickness-Pixels 10 F38 Length-Pixels 11 F39 Length-by-Thick 12 F40Length-by-Width 13 F41 Border-Length-Pixels 14 F7 5a-3D-MacSpic C2.Tumor Shape 15 F13 9b-3D-Circularity (Roundness) 16 F14 9c-3D-Compact 17F23 Asymmetry 18 F24 Compactness 19 F25 Density 20 F26 Elliptic Fit 21F28 Rad-Largest-Enclosed-Ellipse 22 F29 Rad-Smallest-Enclosed-Ellipse 23F30 Shape-Index 24 F31 Roundness 25 F32 Rectangular Fit 26 F88a-3D-Attached to Pleural C3.. Tumor Location 27 F9 8b-3D-Border-to-Lung28 F10 8c-3D-Border-to-Pleural 29 F11 8d-3D-Ratio-Free-to-Attach 30 F129a-3D-FractionalAnisotropy 31 F15 9d-3D-AV-Dist-COG-to-Border 32 F169e-3D-SD-Dist-COG-to-Border 33 F17 9f-3D-Min-Dist-COG-to-Border 34 F189g-3D-Max-Dist-COG-to-Border 35 F19 10a-3D-Relative-Vol-Airspaces 36 F2010b-3D-Number-of-AirSpaces 37 F21 10c-3D-Av-Vol-AirSpaces 38 F2210d-3D-SD-Vol-AirSpaces 39 F27 Main-Direction 40 F4 Mean-Hu C4. PixelAttenuation 41 F5 Standard deviation-Hu Histogram 42 F184 Hist-Mean-L143 F185 Hist-Standard deviation-L1 44 F186 Hist-Energy-L1 45 F187Hist-Entropy-L1 46 F188 Hist-Kurtosis-L1 47 F189 Hist-Skewness-L1 48 F42Average CoOccurrence-Homo C5. Runlength and 49 F43 AverageCoOccurrence-Mp Cooccurrence 50 F44 Average CoOccurrence-Constrast 51F45 Average CoOccurrence-Energy 52 F46 Average CoOccurrence-Entropy 53F47 Average CoOccurrence-Mean 54 F48 AvgGLN 55 F49 AvgHGRE 56 F50AvgLGRE 57 F51 AvgLRE 58 F52 AvgLRHGE 59 F53 AvgLRLGE 60 F54 AvgRLN 61F55 AvgRP 62 F56 AvgSRE 63 F57 AvgSRHGE 64 F58 AvgSRLGE 59 F59 3D-Laws-1(E5 E5 E5 Layer 1) C6. Laws Texture Feature 60 F60 3D-Laws-2 (E5 E5 L5Layer 1) (with different 61 F61 3D-Laws-3 (E5 E5 R5 Layer 1) convolutionfilters) 62 F62 3D-Laws-4 (E5 E5 S5 Layer 1) 63 F63 3D-Laws-5 (E5 E5 W5Layer 1) 64 F64 3D-Laws-6 (E5 L5 E5 Layer 1) 65 F65 3D-Laws-7 (E5 L5 L5Layer 1) 66 F66 3D-Laws-8 (E5 L5 R5 Layer 1) 67 F67 3D-Laws-9 (E5 L5 S5Layer 1) 68 F68 3D-Laws-10 (E5 L5 W5 Layer 1) 69 F69 3D-Laws-11 (E5 R5E5 Layer 1) 70 F70 3D-Laws-12 (E5 R5 L5 Layer 1) 71 F71 3D-Laws-13 (E5R5 R5 Layer 1) 72 F72 3D-Laws-14 (E5 R5 S5 Layer 1) 73 F73 3D-Laws-15(E5 R5 W5 Layer 1) 74 F74 3D-Laws-16 (E5 S5 E5 Layer 1) 75 F753D-Laws-17 (E5 S5 L5 Layer 1) 76 F76 3D-Laws-18 (E5 S5 R5 Layer 1) 77F77 3D-Laws-19 (E5 S5 S5 Layer 1) 78 F78 3D-Laws-20 (E5 S5 W5 Layer 1)79 F79 3D-Laws-21 (E5 W5 E5 Layer 1) 80 F80 3D-Laws-22 (E5 W5 L5Layer 1) 81 F81 3D-Laws-23 (E5 W5 R5 Layer 1) 82 F82 3D-Laws-24 (E5 W5S5 Layer 1) 83 F83 3D-Laws-25 (E5 W5 W5 Layer 1) 84 F84 3D-Laws-26 (L5E5 E5 Layer 1) 85 F85 3D-Laws-27 (L5 E5 L5 Layer 1) 86 F86 3D-Laws-28(L5 E5 R5 Layer 1) 87 F87 3D-Laws-29 (L5 E5 S5 Layer 1) 88 F883D-Laws-30 (L5 E5 W5 Layer 1) 89 F89 3D-Laws-31 (L5 L5 E5 Layer 1) 90F90 3D-Laws-32 (L5 L5 L5 Layer 1) 91 F91 3D-Laws-33 (L5 L5 R5 Layer 1)92 F92 3D-Laws-34 (L5 L5 S5 Layer 1) 93 F93 3D-Laws-35 (L5 L5 W5Layer 1) 94 F94 3D-Laws-36 (L5 R5 E5 Layer 1) 95 F95 3D-Laws-37 (L5 R5L5 Layer 1) 96 F96 3D-Laws-38 (L5 R5 R5 Layer 1) 97 F97 3D-Laws-39 (L5R5 S5 Layer 1) 98 F98 3D-Laws-40 (L5 R5 W5 Layer 1) 99 F99 3D-Laws-41(L5 S5 E5 Layer 1) 100 F100 3D-Laws-42 (L5 S5 L5 Layer 1) 101 F1013D-Laws-43 (L5 S5 R5 Layer 1) 102 F102 3D-Laws-44 (L5 S5 S5 Layer 1) 103F103 3D-Laws-45 (L5 S5 W5 Layer 1) 104 F104 3D-Laws-46 (L5 W5 E5Layer 1) 105 F105 3D-Laws-47 (L5 W5 L5 Layer 1) 106 F106 3D-Laws-48 (L5W5 R5 Layer 1) 107 F107 3D-Laws-49 (L5 W5 S5 Layer 1) 108 F1083D-Laws-50 (L5 W5 W5 Layer 1) 109 F109 3D-Laws-51 (R5 E5 E5 Layer 1) 110F110 3D-Laws-52 (R5 E5 L5 Layer 1) 111 F111 3D-Laws-53 (R5 E5 R5Layer 1) 112 F112 3D-Laws-54 (R5 E5 S5 Layer 1) 113 F113 3D-Laws-55 (R5E5 W5 Layer 1) 114 F114 3D-Laws-56 (R5 L5 E5 Layer 1) 115 F1153D-Laws-57 (R5 L5 L5 Layer 1) 116 F116 3D-Laws-58 (R5 L5 R5 Layer 1) 117F117 3D-Laws-59 (R5 L5 S5 Layer 1) 118 F118 3D-Laws-60 (R5 L5 W5Layer 1) 119 F119 3D-Laws-70 (R5 R5 E5 Layer 1) 120 F120 3D-Laws-71 (R5R5 L5 Layer 1) 121 F121 3D-Laws-72 (R5 R5 R5 Layer 1) 122 F1223D-Laws-73 (R5 R5 S5 Layer 1) 123 F123 3D-Laws-74 (R5 R5 W5 Layer 1) 124F124 3D-Laws-75 (R5 S5 E5 Layer 1) 125 F125 3D-Laws-76 (R5 S5 L5Layer 1) 126 F126 3D-Laws-77 (R5 S5 R5 Layer 1) 127 F127 3D-Laws-78 (R5S5 S5 Layer 1) 128 F128 3D-Laws-79 (R5 S5 W5 Layer 1) 129 F1293D-Laws-80 (R5 W5 E5 Layer 1) 130 F130 3D-Laws-81 (R5 W5 L5 Layer 1) 131F131 3D-Laws-82 (R5 W5 R5 Layer 1) 132 F132 3D-Laws-83 (R5 W5 S5Layer 1) 133 F133 3D-Laws-84 (R5 W5 W5 Layer 1) 134 F134 3D-Laws-85 (S5E5 E5 Layer 1) 135 F135 3D-Laws-86 (S5 E5 L5 Layer 1) 136 F1363D-Laws-87 (S5 E5 R5 Layer 1) 137 F137 3D-Laws-88 (S5 E5 S5 Layer 1) 138F138 3D-Laws-89 (S5 E5 W5 Layer 1) 139 F139 3D-Laws-90 (S5 L5 E5Layer 1) 140 F140 3D-Laws-91 (S5 L5 L5 Layer 1) 141 F141 3D-Laws-92 (S5L5 R5 Layer 1) 142 F142 3D-Laws-93 (S5 L5 S5 Layer 1) 143 F1433D-Laws-94 (S5 L5 W5 Layer 1) 144 F144 3D-Laws-95 (S5 R5 E5 Layer 1) 145F145 3D-Laws-96 (S5 R5 L5 Layer 1) 146 F146 3D-Laws-97 (S5 R5 R5Layer 1) 147 F147 3D-Laws-98 (S5 R5 S5 Layer 1) 148 F148 3D-Laws-99 (S5R5 W5 Layer 1) 149 F149 3D-Laws-100 (S5 S5 E5 Layer 1) 150 F1503D-Laws-101 (S5 S5 L5 Layer 1) 151 F151 3D-Laws-102 (S5 S5 R5 Layer 1)152 F152 3D-Laws-103 (S5 S5 S5 Layer 1) 153 F153 3D-Laws-104 (S5 S5 W5Layer 1) 154 F154 3D-Laws-105 (S5 W5 E5 Layer 1) 155 F155 3D-Laws-106(S5 W5 L5 Layer 1) 156 F156 3D-Laws-107 (S5 W5 R5 Layer 1) 157 F1573D-Laws-108 (S5 W5 S5 Layer 1) 158 F158 3D-Laws-109 (S5 W5 W5 Layer 1)159 F159 3D-Laws-110 (W5 E5 E5 Layer 1) 160 F160 3D-Laws-111 (W5 E5 L5Layer 1) 161 F161 3D-Laws-112 (W5 E5 R5 Layer 1) 162 F162 3D-Laws-113(W5 E5 S5 Layer 1) 163 F163 3D-Laws-114 (W5 E5 W5 Layer 1) 164 F1643D-Laws-115 (W5 L5 E5 Layer 1) 165 F165 3D-Laws-116 (W5 L5 L5 Layer 1)166 F166 3D-Laws-117 (W5 L5 R5 Layer 1) 167 F167 3D-Laws-118 (W5 L5 S5Layer 1) 168 F168 3D-Laws-119 (W5 L5 W5 Layer 1) 169 F169 3D-Laws-120(W5 R5 E5 Layer 1) 170 F170 3D-Laws-121 (W5 R5 L5 Layer 1) 171 F1713D-Laws-122 (W5 R5 R5 Layer 1) 172 F172 3D-Laws-123 (W5 R5 S5 Layer 1)173 F173 3D-Laws-124 (W5 S5 E5 Layer 1) 174 F174 3D-Laws-125 (W5 S5 L5Layer 1) 175 F175 3D-Laws-126 (W5 R5 W5 Layer 1) 176 F176 3D-Laws-127(W5 S5 R5 Layer 1) 177 F177 3D-Laws-128 (W5 S5 S5 Layer 1) 178 F1783D-Laws-129 (W5 S5 W5 Layer 1) 179 F179 3D-Laws-130 (W5 W5 E5 Layer 1)180 F180 3D-Laws-131 (W5 W5 L5 Layer 1) 181 F181 3D-Laws-132 (W5 W5 R5Layer 1) 182 F182 3D-Laws-133 (W5 W5 S5 Layer 1) 183 F183 3D-Laws-134(W5 W5 W5 Layer 1) 190 F190 3D-Wave-1 (P2 L2 C9 Layer 1) C7. WaveletsTexture 191 F191 3D-Wave-2 (P1 L2 C9 Layer 1) (feature at different 192F192 3D-Wave-3 (P2 L2 C10 Layer 1) layers) 193 F193 3D-Wave-4 (P2 L2 C11Layer 1) 194 F194 3D-Wave-5 (P2 L2 C12 Layer 1) 195 F195 3D-Wave-6 (P2L2 C13 Layer 1) 196 F196 3D-Wave-7 (P2 L2 C14 Layer 1) 197 F1973D-Wave-8 (P2 L2 C15 Layer 1) 198 F198 3D-Wave-9 (P2 L2 C1 Layer 1) 199F199 3D-Wave-10 (P2 L2 C2 Layer 1) 200 F200 3D-Wave-11 (P2 L2 C3Layer 1) 201 F201 3D-Wave-12 (P2 L2 C4 Layer 1) 202 F202 3D-Wave-13 (P2L2 C5 Layer 1) 203 F203 3D-Wave-14 (P2 L2 C6 Layer 1) 204 F2043D-Wave-15 (P2 L2 C7 Layer 1) 205 F205 3D-Wave-16 (P2 L2 C8 Layer 1) 206F206 3D-Wave-17 (P1 L2 C11 Layer 1) 207 F207 3D-Wave-18 (P1 L2 C10Layer 1) 208 F208 3D-Wave-19 (P1 L2 C12 Layer 1) 209 F209 3D-Wave-20 (P1L2 C13 Layer 1) 210 F210 3D-Wave-21 (P1 L2 C14 Layer 1) 211 F2113D-Wave-22 (P1 L2 C15 Layer 1) 212 F212 3D-Wave-23 (P1 L2 C1 Layer 1)213 F213 3D-Wave-24 (P1 L2 C2 Layer 1) 214 F214 3D-Wave-25 (P1 L2 C3Layer 1) 215 F215 3D-Wave-26 (P1 L2 C4 Layer 1) 216 F216 3D-Wave-27 (P1L2 C5 Layer 1) 217 F217 3D-Wave-28 (P1 L2 C6 Layer 1) 218 F2183D-Wave-29 (P1 L2 C7 Layer 1) 219 F219 3D-Wave-30 (P1 L2 C8 Layer 1)*See description in (1)

At 108, the tumor object is classified based on the extractedquantitative features. Optionally, classifying the tumor object based onthe extracted quantitative features is predicting whether the tumorobject is a malignant or benign tumor. Alternatively or additionally,the tumor object can be classified using a decision tree algorithm, anearest neighbor algorithm or a support vector machine. It should beunderstood that these classification algorithms are provided only asexamples and that other known classification algorithms can be used.

Optionally, it is possible to reduce the extracted quantitative featuresto a subset of extracted quantitative features, and then classify thetumor object based on the subset of extracted quantitative features. Forexample, the subset of extracted quantitative features can optionallyinclude one or more quantitative features that are predictive of theclassification of the tumor object. In other words, instead ofclassifying the tumor object based on the full set of extractedquantitative features, it is possible to classify the tumor object on asubset of extracted quantitative features having predictive value. Thisreduction of extracted quantitative features is described in detailbelow. Alternatively or additionally, the subset of extractedquantitative features can optionally include one or more non-redundantquantitative features having adequate reproducibility and dynamic range.In other words, it is possible to reduce the extracted quantitativefeatures by eliminating quantitative features having lowerreproducibility metrics and/or lower dynamic range, as well as redundantquantitative features. This reduction of extracted quantitative featuresis also described in detail below.

It should be appreciated that the logical operations described hereinwith respect to the various figures may be implemented (1) as a sequenceof computer implemented acts or program modules (i.e., software) runningon a computing device, (2) as interconnected machine logic circuits orcircuit modules (i.e., hardware) within the computing device and/or (3)a combination of software and hardware of the computing device. Thus,the logical operations discussed herein are not limited to any specificcombination of hardware and software. The implementation is a matter ofchoice dependent on the performance and other requirements of thecomputing device. Accordingly, the logical operations described hereinare referred to variously as operations, structural devices, acts, ormodules. These operations, structural devices, acts and modules may beimplemented in software, in firmware, in special purpose digital logic,and any combination thereof. It should also be appreciated that more orfewer operations may be performed than shown in the figures anddescribed herein. These operations may also be performed in a differentorder than those described herein.

When the logical operations described herein are implemented insoftware, the process may execute on any type of computing architectureor platform. For example, referring to FIG. 4, an example computingdevice upon which embodiments of the invention may be implemented isillustrated. The computing device 400 may include a bus or othercommunication mechanism for communicating information among variouscomponents of the computing device 400. In its most basic configuration,computing device 400 typically includes at least one processing unit 406and system memory 404. Depending on the exact configuration and type ofcomputing device, system memory 404 may be volatile (such as randomaccess memory (RAM)), non-volatile (such as read-only memory (ROM),flash memory, etc.), or some combination of the two. This most basicconfiguration is illustrated in FIG. 4 by dashed line 402. Theprocessing unit 406 may be a standard programmable processor thatperforms arithmetic and logic operations necessary for operation of thecomputing device 400.

Computing device 400 may have additional features/functionality. Forexample, computing device 400 may include additional storage such asremovable storage 408 and non-removable storage 410 including, but notlimited to, magnetic or optical disks or tapes. Computing device 400 mayalso contain network connection(s) 416 that allow the device tocommunicate with other devices. Computing device 400 may also have inputdevice(s) 414 such as a keyboard, mouse, touch screen, etc. Outputdevice(s) 412 such as a display, speakers, printer, etc. may also beincluded. The additional devices may be connected to the bus in order tofacilitate communication of data among the components of the computingdevice 400. All these devices are well known in the art and need not bediscussed at length here.

The processing unit 406 may be configured to execute program codeencoded in tangible, computer-readable media. Computer-readable mediarefers to any media that is capable of providing data that causes thecomputing device 400 (i.e., a machine) to operate in a particularfashion. Various computer-readable media may be utilized to provideinstructions to the processing unit 406 for execution. Common forms ofcomputer-readable media include, for example, magnetic media, opticalmedia, physical media, memory chips or cartridges, a carrier wave, orany other medium from which a computer can read. Examplecomputer-readable media may include, but is not limited to, volatilemedia, non-volatile media and transmission media. Volatile andnon-volatile media may be implemented in any method or technology forstorage of information such as computer readable instructions, datastructures, program modules or other data and common forms are discussedin detail below. Transmission media may include coaxial cables, copperwires and/or fiber optic cables, as well as acoustic or light waves,such as those generated during radio-wave and infra-red datacommunication. Example tangible, computer-readable recording mediainclude, but are not limited to, an integrated circuit (e.g.,field-programmable gate array or application-specific IC), a hard disk,an optical disk, a magneto-optical disk, a floppy disk, a magnetic tape,a holographic storage medium, a solid-state device, RAM, ROM,electrically erasable program read-only memory (EEPROM), flash memory orother memory technology, CD-ROM, digital versatile disks (DVD) or otheroptical storage, magnetic cassettes, magnetic tape, magnetic diskstorage or other magnetic storage devices.

In an example implementation, the processing unit 406 may executeprogram code stored in the system memory 404. For example, the bus maycarry data to the system memory 404, from which the processing unit 406receives and executes instructions. The data received by the systemmemory 404 may optionally be stored on the removable storage 408 or thenon-removable storage 410 before or after execution by the processingunit 406.

Computing device 400 typically includes a variety of computer-readablemedia. Computer-readable media can be any available media that can beaccessed by device 400 and includes both volatile and non-volatilemedia, removable and non-removable media. Computer storage media includevolatile and non-volatile, and removable and non-removable mediaimplemented in any method or technology for storage of information suchas computer readable instructions, data structures, program modules orother data. System memory 404, removable storage 408, and non-removablestorage 410 are all examples of computer storage media. Computer storagemedia include, but are not limited to, RAM, ROM, electrically erasableprogram read-only memory (EEPROM), flash memory or other memorytechnology, CD-ROM, digital versatile disks (DVD) or other opticalstorage, magnetic cassettes, magnetic tape, magnetic disk storage orother magnetic storage devices, or any other medium which can be used tostore the desired information and which can be accessed by computingdevice 400. Any such computer storage media may be part of computingdevice 400.

It should be understood that the various techniques described herein maybe implemented in connection with hardware or software or, whereappropriate, with a combination thereof. Thus, the methods andapparatuses of the presently disclosed subject matter, or certainaspects or portions thereof, may take the form of program code (i.e.,instructions) embodied in tangible media, such as floppy diskettes,CD-ROMs, hard drives, or any other machine-readable storage mediumwherein, when the program code is loaded into and executed by a machine,such as a computing device, the machine becomes an apparatus forpracticing the presently disclosed subject matter. In the case ofprogram code execution on programmable computers, the computing devicegenerally includes a processor, a storage medium readable by theprocessor (including volatile and non-volatile memory and/or storageelements), at least one input device, and at least one output device.One or more programs may implement or utilize the processes described inconnection with the presently disclosed subject matter, e.g., throughthe use of an application programming interface (API), reusablecontrols, or the like. Such programs may be implemented in a high levelprocedural or object-oriented programming language to communicate with acomputer system. However, the program(s) can be implemented in assemblyor machine language, if desired. In any case, the language may be acompiled or interpreted language and it may be combined with hardwareimplementations.

EXAMPLES Methods

As described above, the process of diagnosis consists of lung fieldsegmentation, pulmonary nodule segmentation, feature extraction, andclassification is shown in FIG. 1. The CT images were acquired from theLung Image Database Consortium (LIDC) in DICOM format. Lung FieldSegmentation was accomplished by using the DEFINIENS LUNG TUMOR ANALYSISsoftware suite of DEFINIENS AG of MUNICH, GERMANY. Initial nodulesegmentation was also accomplished by using the DEFINIENS LUNG TUMORANALYSIS software suite. Final segmentation and feature extraction wasperformed using the Single Click Ensemble Segmentation Algorithm, whichis describe in Y. Gu et al., “Automated delineation of lung tumors fromCT images using a single click ensemble segmentation approach,” PatternRecognition, vol. 46, no. 3, pp. 692-702 (2013), available athttp://www.sciencedirect.com/science/article/pii/S0031320312004384.Feature selection was performed using the Weka toolkit, which is a suitof machine learning software developed at the University of Waikato, NewZealand and available at http://www.cs.waikato.ac.nz/ml/index.html, anda correlated feature selection algorithm. Predictive Models wereconstructed using 10-fold cross validation to create training and testdata sets. Overall accuracy, sensitivity, and specificity were recordedalong with Area under the curve (AUC). The following sections furtherdescribe the workflow process.

CT Image Data Set

The CT images were taken from the Lung Image Database Consortium andImage Database Resource Initiative (LIDC/IDRI) image collections; thesecollections are publicly available databases containing lung cancerdiagnostic and screening images from thoracic CT.

As used herein, “low dose” is a threshold of 40 milliampere-seconds(mAs) for this study. Additionally, only cases with a known diagnosiswere selected; cases where it is unclear which nodule is being diagnosedwere eliminated. This yields a total of 33 cases that meet theappropriate criteria. Of these cases, 14 were malignant and 19 werebenign.

Segmentation and Feature Extraction

The lung field segmentation was performed by the automated organsegmentation procedure in the Lung Tumor Analysis (LuTA) software suite,which is a part of DEFINIENS LUNG TUMOR ANALYSIS software suite.Preliminary nodule segmentation was performed with the LuTA softwareusing the published nodule location as the initial seed in a regiongrowing algorithm. Final segmentation and 3D reconstruction wasaccomplished using a Single Click Ensemble Segmentation Algorithm asdescribed above. From the segmented nodule regions, 219 2D and 3D imagefeatures were extracted which include texture features, intensity-basedfeatures, morphological features, and geometric features. An examplesegmented ROI 202 in a radiological image is shown in FIG. 2. Inparticular, the segmented ROI 202 is a segmented lung field in FIG. 2.The lung field has been demarcated by white boundary lines forvisualization. An example segmented tumor object 302 in a radiologicalimage is shown in FIG. 3. In particular, the segmented tumor object 302is a segmented pulmonary nodule in FIG. 3. The pulmonary nodule has beenbounded by white lines for visualization.

Classifiers and Feature Selection

Several commonly used classifiers were tested with texture features,shape features and with a combination of texture and shape features. Theclassifiers were evaluated through 10-fold (10-fold CV)cross-validation. The classification accuracy, sensitivity, andspecificity were recorded for analysis.

1) Decision Tree: A decision tree is a predictive model where nodes aretests conducted on a single data element and leaves are classindicators. The decision tree classifiers used were J48 (Wekaimplementation of C4.5 Decision Tree) and the Weka implementation ofRandom Forests.

2) Nearest Neighbor: A nearest neighbor algorithm is an example ofinstance based learning where classification is determined by the mostcommon class among the nearest neighbors. The Weka implementation of thek-Nearest-Neighbor algorithm (IBk) was used, with k=5.

3) Support Vector Machine: A support vector machine is a supervisedlearning model that non-linearly maps input data to a higher dimensionfeature space. An example SVM implementation is described in C.-C. Changand C.-J. Lin, “Libsvm: A library for support vector machines,” ACMTrans. Intell. Syst. Technol., vol. 2, no. 3, pp. 27:1-27:27, May 2011,available at http://doi.acm.org/10.1145/1961189.1961199. Tuning of theRadial Basis Function kernel used in the SVM was accomplished by using agrid search to provide optimized gamma and cost parameters.

4) Rule Based Classification: The rule based classifier used was Weka'sJRIP, an implementation of the RIPPER algorithm by Cohen. This algorithmconsists of two stages. In the grow phase, a rule is extended greedilyby adding antecedents until the rule has perfect accuracy. Then in theprune phase the rule is pruned by removing antecedent conditions basedon a metric and a pruning data set. Growing and pruning are repeatedwhile there are positive examples or until the error rate exceeds 50%.Finally, rules that would add to the description length are deleted.

5) Naive Bayes: The Naive Bayes classifier, is designed to be used whenfeatures are independent of one another within each class. However, ithas been shown that it often works well even when the features are notindependent. The Naive Bayes classifier estimates the parameters of aprobability distribution given the class, assuming features areconditionally independent. It then computes the posterior probability ofa sample belonging to each class and puts the test sample into the classwith the largest posterior probability. The assumption ofclass-conditional independence greatly simplifies the training step.Even though the class-conditional independence between features does nothold for most data sets, this optimistic assumption works well inpractice. The implementation for Naive Bayes used for this work was fromWeka.

6) Support Vector Machines: Support vector machines are based onstatistical learning theory developed by Cortes and Vapnik and have beenshown by Kramer et al., among others, to obtain high accuracy on adiverse range of application domains such as the letter, page, pendigit,satimage, and waveform data sets. SVMs map the input data to a higherdimensional feature space and construct a hyperplane to maximize themargin between classes. A linear decision surface is constructed in thisfeature space. The hyperplane construction can be reduced to a quadraticoptimization problem; subsets of training patterns that lie on themargin were termed support vectors by Cortes and Vapnik. The formulationused herein allows for “errors” to be on the wrong side of the decisionborder during training. A cost parameter C is multiplied by the distancethe errant example is from the decision boundary. The larger the valueof C the larger the penalty applied in the learning process. Differentkernels, such as a linear kernel, radial basis function kernel, andsigmoid kernel, can be chosen for SVMs. The linear kernel is usedherein. Dehmeshki et al. used support vector machines effectively onCT-scan image data of the lungs in a Computer-Assisted Detection (CAD)system for automated pulmonary nodule detection in thoracic CT-scanimages. The support vector machine libSVM by Chang and Lin was used.Parameter tuning of the cost parameter was conducted on training datausing a grid search after feature selection.

7) Feature Selection: To avoid over-fitting of the data, primary featureset reduction was performed using the Weka implementation of RecursiveElimination of Features (Relief-F). Relief-F uses a nearest-neighborapproach to find features that distinguish between similar examples ofdiffering classes.

Computed image features can have a high correlation with each other.This property combined with the fact that the number of featuresavailable to us was much greater than the number of examples requiredthe investigation of feature selection techniques to improveclassification accuracy. Feature selection was done per fold.Leave-one-out cross validation (LOO) was conducted on the data. Inaddition to feature selection, some of the classifiers' models doimplicit feature selection. For instance, the decision tree and rulebased classifiers subselect features. Also, support vector machinesweight features. However, Naive Bayes uses all provided features forclassification of the test set. All of classifiers explore all of thefeatures to build models on the training set.

1) All Features: This group includes all 219-image features. No featureselection was performed, thus providing a baseline for the effectivenessof the feature selection techniques.

2) Relief-F: The Relief-F algorithm is a feature evaluator that comparesan instance's feature value to the nearest neighbor of both the same andopposite classes. In this work, Relief-F was used to assign ranks toeach individual feature. The top five and ten features found by thealgorithm as shown in Table A. The top ranked features measure tumorattachment to the wall of the lung.

3) Correlation based Feature Selection (CFS): Correlation based FeatureSelection (CFS) searches for features that correlate to a class but donot correlate with each other. The implementation used was found inWEKA. CFS discretizes attributes for nominal classes. The featureschosen are shown in Table B. It can be seen that CFS prefers texturefeatures with a few shape features when compared to the choices ofRelief-F. Relief-F focuses on pleural wall attachment type features.

4) Test-retest: Test-retest features were determined by comparing thestability of features generated after two different scans of the samepatient fifteen minutes apart [13]. If a feature is repeatable then thetwo subsequent scans should yield a similar value. The tumor wassegmented both manually by a radiologist and with a single clickensemble approach. Different thresholds of correlation were used.Attributes were kept that had a test-retest concordance measured by aconcordance correlation coefficient (CCC) of above 0.85, 0.90, and 0.95.At each correlation threshold different attributes were found using themanual and ensemble segmentation methods as well as the intersection ofboth.

TABLE B1 Feature Selection Count Feature Name Top 5 Relief-f 40X8d_3D_Ratio_Free_To_Attached Top 5 Relief-f 40X8a_3D_Is_Attached_To_Pleural_Wall Top 5 Relief-f 40X8c_3D_Relative_Border_To_PleuralWall Top 5 Relief-f 40X8b_3D_Relative_Border_To_Lung Top 5 Relief-f 40X3D.Wavelet.decomposition...P1.L2.C14.Layer.1 Top 10 Relief-f 40X8d_3D_Ratio_Free_To_Attached Top 10 Relief-f 40X8a_3D_Is_Attached_To_Pleural_Wall Top 10 Relief-f 40X8c_3D_Relative_Border_To_PleuralWall Top 10 Relief-f 40X8b_3D_Relative_Border_To_Lung Top 10 Relief-f 40X3D.Wavelet.decomposition...P1.L2.C14.Layer.1 Top 10 Relief-f 40X3D.Wavelet.decomposition...P1.L2.C10.Layer.1 Top 10 Relief-f 39X3D.Laws.features...E5.S5.R5.Layer.1 Top 10 Relief-f 31X3D.Laws.features...W5.W5.L5.Layer.1 Top 10 Relief-f 21X3D.Laws.features...L5.S5.W5.Layer.1 Top 10 Relief-f 5X3D.Wavelet.decomposition...P1.L2.C9.Layer.1 Top 10 Relief-f 13 avgLRETop 10 Relief-f 11 avgSRE Top 10 Relief-f 7X3D.Laws.features...L5.L5.S5.Layer.1 Top 10 Relief-f 8X3D.Laws.features...W5.E5.L5.Layer.1 Top 10 Relief-f 2X3D.Laws.features...S5.S5.E5.Layer.1 Top 10 Relief-f 2X3D.Laws.features...W5.S5.L5.Layer.1 Top 10 Relief-f 2X3D.Wavelet.decomposition...P1.L2.C11.Layer.1 Top 10 Relief-f 4X3D.Laws.features...S5.S5.W5.Layer.1 Top 10 Relief-f 8X3D.Laws.features...S5.L5.E5.Layer.1 Top 10 Relief-f 3X5a_3D_MacSPic_NumberOf Top 10 Relief-f 2 Histogram.ENERGY.Layer.1 Top10 Relief-f 1 X3D.Laws.features...R5.E5.L5.Layer.1 Top 10 Relief-f 1X3D.Laws.features...E5.S5.W5.Layer.1

TABLE B2 Feature Selection Count Feature Name Top 5 CFS 40X3D.Laws.features...W5.S5.R5.Layer.1 Top 5 CFS 40X3D.Laws.features...W5.S5.W5.Layer.1 Top 5 CFS 13X3D.Laws.features...R5.S5.S5.Layer.1 Top 5 CFS 13X3D.Laws.features...S5.S5.W5.Layer.1 Top 5 CFS 14X3D.Laws.features...W5.S5.S5.Layer.1 Top 5 CFS 28 Longest.Diameter...mm.Top 5 CFS 26 Short.Axis...Longest.Diameter...mm... Top 5 CFS 24Short.Axis...mm. Top 5 CFS 1 X3D.Laws.features...S5.S5.R5.Layer.1 Top 10CFS 1 X3D.Wavelet.decomposition...P1.L2.C4.Layer.1 Top 10 CFS 40X3D.Laws.features...W5.S5.R5.Layer.1 Top 10 CFS 40X3D.Laws.features...W5.S5.W5.Layer.1 Top 10 CFS 13X3D.Laws.features...R5.S5.S5.Layer.1 Top 10 CFS 13X3D.Laws.features...S5.S5.W5.Layer.1 Top 10 CFS 14X3D.Laws.features...W5.S5.S5.Layer.1 Top 10 CFS 40Longest.Diameter...mm. Top 10 CFS 40Short.Axis...Longest.Diameter...mm... Top 10 CFS 40 Short.Axis...mm. Top10 CFS 40 Mean...HU. Top 10 CFS 40 StdDev...HU. Top 10 CFS 28Volume...cm... Top 10 CFS 26 X5a_3D_MacSpic_NumberOf Top 10 CFS 24X8a_3D_Is_Attached_To_Pleural_Wall Top 10 CFS 1X3D.Laws.features...S5.S5.R5.Layer.1 Top 10 CFS 1X3D.Wavelet.decomposition...P1.L2.C4.Layer.1

The results of running Pearson Product-Moment Correlation Coefficient onthe data set indicated that a large number of features had a high degreeof correlation/anti-correlation. Since Relief-F does not calculate thecorrelation between features, the Weka implementation ofCorrelation-based Feature Subset Selection for Machine Learning (CFS)was explored. This algorithm selects features that are predictive of theclass using Pearson's Coefficient while eliminating those that have ahigh degree of correlation with each other (also based on Pearson'sCoefficient). Additionally, an algorithm referred to herein as “Relief-FNon-Correlated” (“RFNC”) (also referred to herein as Relief-F withCorrelation Detection algorithm) that used the Relief-F rankings todetermine class predictability while eliminating correlated featureswhere the Pearson's Coefficient exceeded a specified threshold wasdeveloped.

Feature selection for all experiments was performed on the entire dataset using Leave-One-Volume-Out cross validation resulting in anoptimistic feature set. For Relief-F, the features with the highestaverage merit were selected. For CFS, all features were selected thatappeared in more than 50% of the folds.

Experiments

Segmentation and feature extraction were performed on 33 cases from theLIDC data set, consisting of 14 malignant and 19 benign nodules.Attribute values were normalized −1 to 1. Experiments were conductedusing three separate sets of features. The first set consisted of 47shape features, the second of 172 texture features, and the third is acombined set of 219 features. Training and testing data sets weregenerated using 10-fold cross validation.

For each classifier and data set, experiments were performed using adata set that consisted of all features (no feature selection), featuresets consisting of the top 10 and top 5 Relief-F features, a set of 5features based on CFS (4 features in the case for the shape-feature dataset), and sets of features consisting of the top five non-correlatedRelief-F features with Pearson's Coefficient thresholds of 0.8 and 0.5.

Results

All experiments resulted in accuracies that exceeded guessing themajority class (57.57%). Table I shows the classification accuracyachieved by different classifiers using 47 shape features. The highestaccuracy of 81.82% was achieved using J48 with CFS feature selection (4features).

TABLE I PERFORMANCE OF CLASSIFIERS WITH 47 SHAPE FEATURES ClassifierFeat. Sel. Accuracy Sensitivity Specificity J48 All Features 75.76%71.43% 78.95% Relief-F 78.79% 71.43% 84.21% CFS 81.32% 71.43% 89.47%RFNC .80 72.73% 57.14% 82.21% RFNC .50 75.76% 57.14% 89.47% RandomForest All Features 75.76% 71.43% 78.95% Relief-F  69.7% 64.29% 72.63%CFS 60.61% 42.86% 73.63% RFNC .80 69.70% 71.43% 68.42% RFNC .50 72.73%71.43% 73.68% IBk (5 NN) All Features 78.79% 64.29% 89.47% Relief-F63.64% 42.86% 78.95% CFS 72.73% 50.00% 89.47% RFNC .80 78.79% 64.29%89.47% RFNC .50 75.76% 57.14% 89.47% SVM All Features 63.64% 42.86%78.95% Relief-F 72.73% 50.00% 89.47% CFS 72.73% 57.14% 84.21% RFNC .8069.70% 57.14% 78.95% RFNC .50 73.79% 64.29% 89.47%

Table II shows the results of the same classifiers using only texturefeatures. Accuracy generally increased with a few exceptions. Moreover,the best accuracy of 90.91% was achieved using IBk (5 NN) with CFSfeature selection.

TABLE II PERFORMANCE OF CLASSIFIERS WITH 172 TEXTURE FEATURES ClassifierFeat. Sel. Accuracy Sensitivity Specificity J48 All Features 78.79%71.43% 84.21% Relief-F 81.82% 92.86% 73.68% CFS 66.67% 50.00% 78.95%RFNC .80 60.61% 50.00% 68.42% RFNC .50 60.61% 50.00% 68.42% RandomForest All Features 72.73% 64.29% 78.95% Relief-F 66.67% 57.14% 73.68%CFS 72.73% 57.14% 82.21% RFNC .80 72.73% 64.29% 78.95% RFNC .50 75.76%64.29% 84.21% IBk (5 NN) All Features 78.79% 78.57% 78.95% Relief-F81.82% 78.57% 84.21% CFS 90.91% 85.71% 94.74% RFNC .80 84.85% 78.57%89.47% RFNC .50 81.82% 78.57% 84.21% SVM All Features 84.85% 78.57%89.47% Relief-F 75.76% 78.57% 73.68% CFS 78.79% 71.43% 84.21% RFNC .8081.82% 71.43% 89.47% RFNC .50 78.79% 78.57% 78.95%

Table III shows the results using both texture and shape features.Accuracies generally improved over experiments using only shape ortexture features. The highest accuracy was 87.88% using J48 with nofeature selection and SVM using CFS.

TABLE III PERFORMANCE OF CLASSIFIERS WITH 219 TEXTURE AND SHAPE FEATURESClassifier Feat. Sel. Accuracy Sensitivity Specificity J48 All Features87.88% 85.71% 89.47% Relief-F 81.82% 92.86% 73.68% CFS 69.70% 57.14%78.95% RFNC .80 69.70% 57.14% 78.95% RFNC .50 66.67% 50.00% 78.95%Random Forest All Features 72.73% 57.14% 84.21% Relief-F 66.67% 57.14%73.68% CFS 75.76% 57.14% 89.47% RFNC .80 72.73% 71.43% 73.68% RFNC .5066.67% 57.14% 73.68% IBk (5 NN) All Features 84.85% 78.57% 89.47%Relief-F 81.82% 78.57% 84.21% CFS 81.82% 71.43% 89.47% RFNC .80 87.88%78.57% 94.74% RFNC .50 75.76% 57.14% 89.47% SVM All Features 84.85%78.57% 89.47% Relief-F 75.76% 78.57% 73.68% CFS 87.88% 85.71% 89.47%RFNC .80 87.88% 78.57% 94.74% RFNC .50 78.79% 64.29% 89.47%

Table IV shows the top shape features selected by Relief-F, Table V, thetop texture features, and Table VI, the top features from the combinedfeature set.

TABLE IV TOP RELIEF-F 47 SHAPE FEATURES Feature Name3D_SD_Dist_COG_To_Border 3D_Is_Attached_To_Pleural_Wall3D_MAX_Dist_COG_To_Border Longest Diameter Length

TABLE V TOP RELIEF-F 172 TEXTURE FEATURES Feature Name 3D Waveletdecomposition. P2 L2 C8 Layer 1 3D Wavelet decomposition. P2 L2 C15Layer 1 3D Wavelet decomposition. P2 L2 C1 Layer 1 3D Waveletdecomposition. P2 L2 C7 Layer 1 3D Wavelet decomposition. P2 L2 C4 Layer1

TABLE VI TOP RELIEF-F 219 TEXTURE AND SHAPE FEATURES Feature Name 3DWavelet decomposition P2 L2 C8 Layer 1 3D Wavelet decomposition P2 L2C15 Layer 1 3D Wavelet decomposition P2 L2 C1 Layer 1 3D Waveletdecomposition P2 L2 C7 Layer 1 3D Wavelet decomposition P2 L2 C4 Layer 1

Tables VII, VIII and IX enumerate the features selected by the CFSalgorithm from the shape-feature data set, the texturefeature-only dataset, and the combined data set respectively.

TABLE VII CFS SELECTION FROM 47 SHAPE FEATURES Feature Name LongestDiameter 3D_Relative_Border_To_Lung 3D_MIN_Dist_COG_To_Border EllipticFit

TABLE VIII CFS SELECTION FROM 172 TEXTURE FEATURES Feature NameavgCoocurrence-MP 3D Laws features L5 E5 R5 Layer 1 3D Waveletdecomposition. P2 L2 C1 Layer 1 3D Wavelet decomposition. P1 L2 C1 Layer1 3D Wavelet decomposition. P1 L2 C2 Layer 1

TABLE IX CFS SELECTION FROM 219 TEXTURE AND SHADE FEATURES Feature Name3D Laws features L5 E5 R5 Layer 1 3D Wavelet decomposition. P2 L2 C1Layer 1 3D Wavelet decomposition. P1 L2 C1 Layer 1 3D Waveletdecomposition. P1 L2 C2 Layer 1 Elliptic Fit

Tables X, XII, and XIV list the features selected by the RFNC algorithmwith threshold 0.8 from the shape-feature data set, thetexture-feature-only data set, and the combined data set; tables XI,XIII, and XV list the features selected for a threshold of 0.5.

TABLE X RELIEF-F NON-CORRELATED SELECTION FROM 47 SHAPE FEATURES,THRESHOLD = 0.8 Feature Name 9e_3D_SD_Dist_COG_To_Border_[mm]8d_3D_Ratio_Free_To_Attached Rectangular Fit Shape index HistogramENTROPY Layer 1

TABLE XI RELIEF-F NON-CORRELATED SELECTION FROM 47 SHAPE FEATURES,THRESHOLD = 0.5 Feature Name 9e_3D_SD_Dist_COG_To_Border_[mm]8d_3D_Ratio_Free_To_Attached Histogram ENTROPY Layer 1 Histogram SKEWLayer 1 Asymmetry

TABLE XII RELIEF-F NON-CORRELATED SELECTION FROM 172 TEXTURE FEATURES,THRESHOLD = 0.8 Feature Name 3D Wavelet decomposition. P2 L2 C8 Layer 1avgRP 3D Laws features L5 E5 R5 Layer 1 3D Wavelet decomposition. P1 L2C3 Layer 1 3D Laws features R5 E5 R5 Layer 1

TABLE XIII RELIEF-F NON-CORRELATED SELECTION FROM 172 TEXTURE FEATURES,THRESHOLD = 0.5 Feature Name 3D Wavelet decomposition. P2 L2 C8 Layer 13D Laws features L5 E5 R5 Layer 1 3D Laws features R5 L5 R5 Layer 1 3DLaws features W5 E5 E5 Layer 1 avgCoocurrence-entropy

TABLE XIV RELIEF-F NON-CORRELATED SELECTION FROM 219 TEXTURE AND SHAPEFEATURES, THRESHOLD = 0.8 Feature Name 3D Wavelet decomposition. P2 L2C8 Layer 1 9e_3D_SD_Dist_COG_To_Border_[mm] Rectangular Fit 3D Lawsfeatures L5 E5 R5 Layer 1 3D Wavelet decomposition. P1 L2 C3 Layer 1

TABLE XV RELIEF-F NON-CORRELATED SELECTION FROM 219 TEXTURE AND SHAPEFEATURES, THRESHOLD = 0.5 Feature Name 3D Wavelet decomposition. P2 L2C8 Layer 1 3D Laws features L5 E5 R5 Layer 18a_3D_Is_Attached_To_Pleural_Wall Histogram ENTROPY Layer 1 HistogramSKEW Layer 1

For the shape-feature data set, feature selection yielded the same orbetter accuracies for the J48 and SVM classifiers as compared to nofeature selection. Only IBk (5 NN) showed consistent improvement withfeature selection on the texture-feature-only data set; the most notableimprovement being the 90.91% accuracy reported earlier. Featurereduction on the combined data set produced mixed results.

The Correlation-based Feature Subset Selection (CFS) algorithm was mosteffective in conjunction with the J48 and SVM classifiers on theshape-feature data set and resulted in less effective feature sets inconjunction with the other classifiers. On both the texture and combineddata sets, CFS resulted in effective reduced feature sets when using IBk(5 NN) and SVM.

Conclusion

The positive effect of texture features in the classification ofpulmonary nodules in low-dose CT have been shown. There were 219 imagefeatures extracted which included texture and shape features. Thefeatures were used for the classification of pulmonary nodules intomalignant and benign using several classifiers like Decision Trees,Random Forests, Nearest Neighbors, and Support Vector Machines. Whilethe best overall results were achieved using CFS feature selection andthe IBk classifier with the texture-feature-only data set, moreconsistent results were obtained with the combined data set.

With a combination of texture and shape features, the highest accuracyachieved was 87.88%—an improvement over the 81.82% achieved when onlythe shape features were used. When using the SVM classifier and CFSfeature selection, the sensitivity improved from 71.43% to 85.71% whilethe specificity remained the same at 89.47%. Similarly, the IBk (5 NN)classifier with a non-correlated Relief-F feature set, showed the sameaccuracy improvement with a sensitivity of 78.57% and a specificity of94.74%.

Tables XVI-XXI show additional accuracy results based on featureselection method.

TABLE XVI Avg LQ UQ Classifier Features # Accy Accy Accy AUC DecisionTree Top 5 Relief-f 5 77.5% 65% 90% 0.712 Decision Tree Top 10 Relief-f10   70% 65% 75% 0.732 Rules All 219 62.5% 65% 60% 0.729 Rules All Top 5Relief-f 5   75% 65% 85% 0.661 Naive Bayes All Top 10 Relief-f 10   65%55% 75% 0.52 Naive Bayes Manual & Ensemble 5   60% 45% 75% 0.64test-retest (.85) Top 5 RF SVM Manual test-retest 10   65% 70% 60% 0.65(.90) Top 10 Relief-f SUMMARY OF THE HIGHEST SURVIVAL LEAVE-ONE-OUTACCURACY AND AUC RESULTS CONTAINING THE FEATURE SELECTION METHOD, NUMBEROF FEATURES, AVERAGE ACCURACY, LOWER QUARTILE ACCURACY, UPPER QUARTILEACCURACY, AND THE AREA UNDER THE RECEIVER OPERATING CURVE. LQ IS LOWERQUARTILE AND UQ IS UPPER QUARTILE.

TABLE XVII Avg LQ UQ Classifier Features # Accy Accy Accy AUC DecisionTree Top 5 Relief-f 5 77.5%  65% 90% 0.712 Top 10 Relief-f 10   70%  65%75% 0.732 All Top 5 CFS 5 62.5%  95% 30% 0.292 All Top 10 CFS 10   65% 75% 55% 0.552 Manual test-retest (.95) 45   60%  95% 25% 0.271 Manualtest-retest (.90) Top 10 Relief-f 10 67.5%  70% 65% 0.562 Manualtest-retest (.90) Top 10 CFS 10   60%  70% 50% 0.435 Manual test-retest(.85) Top 5 Relief-f 5 62.5%  85% 40% 0.455 Manual test-retest (.85) Top5 CFS 5 72.5%  95% 50% 0.488 Manual test-retest (.85) Top 10 CFS 1062.5%  70% 55% 0.51 Ensemble test-retest (.95) Top 5 CFS 5   65% 100%30% 0.3 Ensemble test-retest (.95) Top 10 CFS 10   65% 100% 30% 0.3Ensemble test-retest (.90) Top 5 CFS 5 62.5%  95% 30% 0.292 Ensembletest-retest (.90) Top 10 CFS 10   65%  75% 55% 0.524 Ensembletest-retest (.85) Top 5 CFS 5 62.5%  95% 30% 0.292 Ensemble test-retest(.85) Top 10 CFS 10   65%  75% 55% 0.524 Manual & Ensemble test-retest(.90) 10   65%  70% 60% 0.691 Top 10 Relief-f Manual & Ensembletest-retest (.85) 10 62.5%  65% 60% 0.68 Top 10 Relief-f SURVIVALLEAVE-ONE-OUT ACCURACY RESULTS DOING FURTHER FEATURE SELECTION ONTEST-RETEST FEATURES FOR DECISION TREES CONTAINING THE FEATURE SELECTIONMETHOD, NUMBER OF FEATURES, AVERAGE ACCURACY, LOWER QUARTILE ACCURACY,UPPER QUARTILE ACCURACY, AND THE AREA UNDER THE RECEIVER OPERATINGCURVE. THE TOP ACCURACY AND AUC ARE IN BOLD.

TABLE XVIII Avg LQ UQ Classifier Features # Accy Accy Accy AUC Rules All219 62.5% 65% 60% 0.729 All Top 5 Relief-f 5   75% 65% 85% 0.661 All Top10 Relief-f 10   65% 60% 70% 0.598 Manual test-retest (.90) 10 62.5% 75%50% 0.568 Top 10 Relief-f Manual test-retest (.85) 95 62.5% 75% 50%0.688 SURVIVAL LEAVE-ONE-OUT ACCURACY RESULTS DOING FURTHER FEATURESELECTION ON TEST-RETEST FEATURES FOR JRIP CONTAINING THE FEATURESELECTION METHOD, NUMBER OF FEATURES, AVERAGE ACCURACY, LOWER QUARTILEACCURACY, UPPER QUARTILE ACCURACY, AND THE AREA UNDER THE RECEIVEROPERATING CURVE. THE TOP ACCURACY AND AUC ARE IN BOLD.

TABLE XIX Avg LQ UQ Classifier Features # Accy Accy Accy AUC Naive AllTop 5 Relief-f 5 62.5% 45% 80% 0.605 Bayes All Top 10 Relief-f 10   65%55% 75% 0.52 Manual test-retest (.95) 5   60% 40% 80% 0.458 Top 5Relief-f Manual test-retest (.90) 5   60% 45% 75% 0.54 Top 5 Relief-fManual & Ensemble 5   60% 40% 80% 0.552 test-retest (.95) Top 5 Relief-fManual & Ensemble 5   60% 45% 75% 0.64 test-retest (.85) Top 5 RFSURVIVAL LEAVE-ONE-OUT ACCURACY RESULTS DOING FURTHER FEATURE SELECTIONON TEST-RETEST FEATURES FOR NAIVE BAYES CONTAINING THE FEATURE SELECTIONMETHOD, NUMBER OF FEATURES, AVERAGE ACCURACY, LOWER QUARTILE ACCURACY,UPPER QUARTILE ACCURACY, AND THE AREA UNDER THE RECEIVER OPERATINGCURVE. THE TOP ACCURACY AND AUC ARE IN BOLD.

TABLE XX Avg LQ UQ Classifier Features # Accy Accy Accy AUC SVM Manualtest-retest (.90) 10   65% 70% 60% 0.65  Top 10 Relief-f Manual &Ensemble test- 5 62.5% 65% 60% 0.625 retest (.90) Top 5 Relief-f Manual& Ensemble test- 10   60% 65% 55% 0.6  retest (.90) Top 10 Relief-fManual & Ensemble test- 5 62.5% 65% 60% 0.625 retest (.85) Top 5Relief-f Manual & Ensemble test- 10   60% 65% 55% 0.6  retest (.85) Top10 Relief-f SURVIVAL LEAVE-ONE-OUT ACCURACY RESULTS DOING FURTHERFEATURE SELECTION ON TEST-RETEST FEATURES FOR SVM CONTAINING THE FEATURESELECTION METHOD, NUMBER OF FEATURES, AVERAGE ACCURACY, LOWER QUARTILEACCURACY, UPPER QUARTILE ACCURACY, AND THE AREA UNDER THE RECEIVEROPERATING CURVE. THE TOP ACCURACY AND AUC ARE IN BOLD.

TABLE XXI Avg LQ UQ Classifier Features # Accy Accy Accy AUC DecisionTree Volume 1   45% 40% 50% 0.45 Rules Volume 1 32.5% 45% 20% 0.223Naive Bayes Volume 1   45% 60% 30% 0.388 SVM Volume 1   15% 20% 10% 0.15SURVIVAL LEAVE-ONE-OUT ACCURACY RESULTS USING ONLY VOLUME CONTAINING THEFEATURE SELECTION METHOD, NUMBER OF FEATURES, AVERAGE ACCURACY, LOWERQUARTILE ACCURACY, UPPER QUARTILE ACCURACY, AND THE AREA UNDER THERECEIVER OPERATING CURVE. THE TOP ACCURACY AND AUC ARE IN BOLD

Methods—Quantitative Feature Selection

Classically, CT imaging is routinely used to establish anatomical andmacroscopic pathologies in cancer patients. Although not commonly used,CT images of tumors also depict characteristics that can be related tophysiological processes, such as cell density, necrosis and perfusion.The appearance of the tumor in CT images has been used, qualitatively,to provide information about tumor type, degree of spread and organinvasion. Such features are typically described subjectively (e.g.“mildly irregular”, “highly spiculated”, “moderate necrosis”). However,to be useful as biomarkers, features must be reproducible, quantifiableand objective. Thus, there is a need to identify features from CT imagesthat can be reliably extracted and converted into quantifiable, mineabledata as potential prognostic, predictive or response biomarkers. Incurrent clinical practice, only two tumor quantitative CT features—bi-and uni-dimensional measurements (WHO and RECIST, respectively) areroutinely obtained and used to assess response to therapy. While theseare satisfactory under some conditions, reduction in tumor size oftendoes not reflect clinico-pathological response.

Recent advances in both image acquisition and image analysis techniquesallow semi-automated segmentation, extraction and quantization ofnumerous features from images, such as texture. Such features extractedfrom CT images of lung tumors have been shown to relate to glucosemetabolism and stage, distinguish benign from malignant tumors, ordifferentiate between aggressive and nonaggressive malignant lungtumors. In liver cancer, combinations of twenty-eight image featuresobtained from CT images could reconstruct 78% of the global geneexpression profiles. As this area of investigation continues to expand,a number of critical questions remain unanswered, including theredundancy and reproducibility of individual features. In the presentstudy, a large number of image features describing shape, size, runlength encodings, attenuation histograms, textures, entropy, andwavelets are extracted and analyzed. In this agnostic approach, equalimportance is given to all features with no prior bias towardsradiologist preferences or accepted semantics. Such an analysis of ahigh dimensional feature space, i.e. “radiomics”, requiresstandardization and optimization to qualify these potential biomarkersfor prognosis, prediction or therapy response. An important step in thequalification process is to statistically characterize individualfeatures as being reproducible, non-redundant and having a largebiological range. The most reproducible features are more likely to beable to identify subtle changes with time, pathophysiology or inresponse to therapy. Additionally, the reproducibility must be comparedto the entire biological range available to that feature acrosspatients. The biological range relative to reproducibility can beexpressed as a dynamic range (DR). It is expected that features will bemore useful if they have a large dynamic range. In addition, featuresmust be identified that are not redundant, as it is axiomatic thatredundant features can overwhelm clustering algorithms and decisionsupport systems.

The inter-scan reproducibility of features may be affected bydifferences in patient variables, such as positioning, respiration phaseand contrast enhancement, as well as acquisition and processingparameters, including image acquisition power and slice thickness, imagereconstruction algorithm, segmentation software and user input forsegmentation. In the present study, the acquisition and processingparameters were fixed, and patient variables were minimized by obtainingtwo separate CT scans from the same patient on the same machine usingthe same parameters, within 15 minutes of each other. Acquisition ofthese images and reproducibility of tumor uni-dimensional,bi-dimensional and volumetric measurements has been previously reported.These data have been made publically available under the NCI-sponsoredReference Imaging Database to Evaluate Response (RIDER) project. Theobjective of the current study is to determine the variability in alarge set of agnostic image features from this data set in order toidentify the most informative features using empirical filters.

In prior work, it was demonstrated that semi-automatic (e.g., Manual)segmentation had 73% overlap between operators across a test set of 129patients. Hence, lesions in the current study were segmented and 219features were extracted from these segmented volumes. The feature setcovers a broad range of shape, size, and texture type features.

Although the study began with a large feature set compared to priorconventional radiological analyses, it is expected that there may beredundancy in these features due to the sample size and the fact that anumber are in the same family (e.g., texture). Thus, to reduce thedimensionality of this agnostic data set, features were first filteredbased on their reproducibility, e.g., those with the highestintra-feature concordance correlation coefficients (CCC) between therepeats. As a second filter, dynamic range based on inter-patientvariability was used. Finally, redundancy was assessed by computing aninter-feature coefficient of determination (R² _(Bet)) between allpossible pairs of features. A representative feature set was found bycombing dependent groups to form an independent set. These featurescould also be used for prognosis prediction or prediction of progressionor other analysis, a practical application is also provided.

Data Collection

In brief, baseline and follow up CTs of the thorax for each patient wereacquired within 15 minutes of each other (e.g., the fixed time period asused herein), using the same CT scanner and imaging protocol. It shouldbe understood that a fixed time period of 15 minutes is used only as anexample and that fixed time periods more or less than 15 minutes can beused. Among other possibilities, this enables testing extracted imagefeatures for stability. Unenhanced thoracic CT images were acquiredusing 16-detector (GE LightSpeed) or 64-detector (VCT; GE Heathcare)scanners, with 120 kvp tube voltage and image slices thickness of 1.25mm were reconstructed using the same lung convolution kernel withoutoverlap. The CT scans were acquired from 32 patients (mean age, 62.1years; range, 29-82 years) with non-small cell lung cancer. There were16 men (mean age, 61.8 years; range, 29-79 years) and 16 women (meanage, 62.4 years; range, 45-82 years). All patients had a primarypulmonary tumor of 1 cm or larger. These images were deposited in theNational Biomedical Imaging Archive (NBIA) maintained by NIH. The imagesare available in the “RIDER Lung CT” collection in NBIA under the“Collections” sections.

Segmentation of Tumors

DEFINIENS DEVELOPER XD of DEFINIENS AG of MUNICH, GERMANY was used asthe image analysis platform. It is based on the COGNITION NETWORKTECHNOLOGY which allows the development and execution of image analysisapplications. Here, the Lung Tumor Analysis (LuTA) application was used.LuTA contains a semi-automated three-dimensional click-and-grow approachfor segmentation of tumors under the guidance of an operator,hereinafter referred to as “manual” segmentation. The “manual” workflowcontained the following steps: (a) Preprocessing: The preprocessingperformed automated organ segmentation with the main goal of segmentingthe aerated lung with correct identification of the pleural wall inorder to facilitate the semi-automated segmentation of juxtapleurallesions. (b) Semi-automated correction of the pulmonary boundary: Inorder to perform the seed based segmentation of a target lesion, thelatter has to be completely within the extracted lung image object. Incases where a medical expert concluded that the automated preprocessingdescribed above failed to accurately identify the border between atarget lesion and the pleural wall, it was necessary to enablecorrection of the automated lung segmentation. To this end, the imageanalysts identified the part of the lung that needed modification andplaced a seed point manually where the segmentation should be corrected.A seed point outside the lung defined a lung extension, whereas a seedpoint inside the lung defined a reduction. (c) Click and Grow: In orderto segment a target lesion the image analysts identified the lesionwithin the segmented lung and placed a seed point in itsinterior—typically at the perceived center of the lesion. If the growingprocess did not sufficiently capture the target lesion, the operatorcould place additional seed points within the lesion and repeat thegrowing process outlined above. Upon completion of the segmentation, theindividual image objects were merged to form a single image objectrepresenting the segmented target lesion. (d) Manual refinement andgeneration of lesion statistics: Upon completing a seed based lesionsegmentation as described above; the results were viewed by scrolling upand down the stacks of axial images to verify that the segmentationfollowed the anatomical compartment boundaries properly. To facilitatemanual adjustment of the seed based growing algorithm, tools of twotypes were constructed. The first type allowed the operator to limit theboundaries beyond which the region could grow during the “Click-Grow”step by manually placing “blocker” points. Another approach allowed formanual editing of the contour of each segmented lesion on each axialslice by cutting, merging and reclassifying objects and thus enabled theimage analysts to perform any desired modifications of the segmentedlesion. Image analysts were empowered to override as much or as littleof the semi-automatically grown regions as their expertise suggested wasindicated. The “manual” segmentation (MS) process (a)-(d) above requiredmultiple human interactions in order to get the ‘correct’ segmentationboundaries.

Once the segmentation of all target lesions was complete, statistics foreach lesion, such as volume, center of gravity and average density, allreadily available as object features within the commercial cognitivenetwork language (CNL), were extracted. In total, 64 lesions weresegmented, i.e. 2 per patient. Then quantitative values of imagefeatures were extracted from each segmented volume.

Image Features

Several types of image features were extracted to describe the tumorsheterogeneous shape and structure (details in the subsection below). Itis to be noted that there are multiple features extracted in some of thecategories. As described above, texture features are good descriptors ofthe tumor and have shown relevance for survival prediction. In thisstudy, 219 custom 3-dimensional image features were used. The featuresdetails are described in Tables A1 and A2 above. Most size and shapebased feature computation were implemented within the DEFINIENSDEVELOPER XD platform, while texture and other derived features werecomputed from algorithms implemented in C/C++. All the features wereobtained from the region of interest (e.g., after the segmentation).

Feature Categorization

The agnostic features types were assembled to describe the tumor lesion,though 219 features seems like a large set may other effectivedescriptors may yet need to be added. The feature set is categorizedinto seven broad categories to describe the lesion, namely: size based,shape based, location based, Pixel histogram intensity based, Run length& Co-occurrence, Law's kernel based texture and Wavelets based texturedescriptors. Tables A1 and A2 shows the number of features in each ofthe categories. The approach has been driven by the conventionalradiologist belief that a heterogeneous tumor lesion is best describedby an ensemble of factors ranging from tumors' shape, size, location,and density. It has also been shown that features are dependent withinand across the categories. The approach is to find representativefeatures in each category so as to “best describe” the tumor in featurespace. Comprehensive descriptors to cover most categories have beenassembled.

Feature Selection Procedure

The sets of informative features were selected using a three stepprocess. First, the consistency between the Test and Re-Test experimentswere tested. For each image feature the concordance correlationcoefficient was used to quantify reproducibility between two scansperformed on each patient. The Concordance correlation coefficient issuperior to the Pearson correlation coefficient for repeatedexperiments. If X_(1,k) and X_(2,k) are the feature values for k^(th)feature, assuming (X_(1,k)(i), X_(2,k)(i)) are independent and follow abivariate distribution with means and covariance matrix: μ_(x1), μ_(x2)and ([σ_(x1,k) ², σ_(x1,k, x2,k) ²], [σ_(x1,k, x2,k) ², σ_(x2,k) ²]),for the lesions measured in the i^(th) test and retest experiment. Thenthe Concordance coefficient (CCC) is given by the following Eqn. (1),

$\begin{matrix}{{CCC} = \frac{\left( {2\; \sigma_{x_{1,k},x_{2,k}}} \right)}{\left( {\sigma_{x_{1,k}}^{2} + \sigma_{x_{2,k}}^{2} + \left\lbrack {\mu_{x_{1,k}} - \mu_{x_{2,k}}} \right\rbrack^{2}} \right)}} & (1)\end{matrix}$

The CCC evaluates the degree to which the experimental samples arelocated on the 45° line through the origin in a plot spanned by the twomeasurements X_(1,k) & X_(2,k). The concordance correlation is typicallyused to measure the deviation in repeated experiments because of itsability to measure deviation from the best fit of the data to the 45°line through the origin. The CCC values ranges from 1 to −1, implyingperfect agreement between the repeated experiments to reverse agreementbetween them.

On this set of highly reproducible features, the next step was to selectthe features with a large inter-patient variability, using the “dynamicrange”. The normalized dynamic range for a feature was defined as theinverse of the average difference between measurements divided by theentire range of observed values in the sample set as in Eqn. (2):

$\begin{matrix}{{DR}_{k} = \left( {1 - {{1/n}{\sum\limits_{i = 1}^{n}\; \frac{{{{Test}_{k}(i)} - {{ReTest}_{k}(i)}}}{{Max}_{k} - {Min}_{k}}}}} \right)} & (2)\end{matrix}$

where i refers to sample index, for the k^(th) feature, Test_(k)(i) orReTest_(k)(i) are sample i's k^(th) feature values for a Test/ReTestpopulation of n patient cases, the maximum (Max_(k)) and minimum(Min_(k)) are computed on the entire sample set. The dynamic range forfeature k has a range, DR_(k)∈[0,1]. Values close to 1 are preferred,and imply that the feature has a large biological range relative toreproducibility. Increasing the variation between the Test-Retestrepeats will lead to a reduction in the DR value. Screening for a largeDR will eliminate features that show greater variability in the repeatscans compared to the range of the coverage. The dynamic range measurewill effectively address the ‘effect size’ by identifying features witha lower value that are either not reproducible (relative to theirrange), or that are not highly variable across an entire sample set.This metric helps to score features with a higher score for one thatshows relatively larger coverage with respect to the repeatabledifferences, this does not intend to describe the dynamic range of theentire population.

The last step is to eliminate redundancies, based on the calculation ofdependencies within the group. The coefficient of determination (R²_(Bet)) between the features that passed a dynamic range threshold toquantify dependency were calculated. It is a linear estimate of thecorrelation or dependency and has a range of 0 to 1. Values close to 1would mean that the data points are close to the fitted line (i.e.closer to dependency). The coefficient of determination of simpleregression is equal to the square of the Pearson correlationcoefficient. Different threshold values for R² _(Bet) were used toconsider each feature as linearly dependent on any other feature(s) inthe list. The features that passed the cutoff limit were grouped andreplaced by a representative from the group; the one having highestdynamic range. The purpose of this third filter was to eliminateredundancies (and not necessarily identify independence). A range of R²_(Bet) thresholds were explored. The features category were filtered tofind the representative in each category to ensure the final set offeatures have descriptors from each group. Feature reduction taking allthe categories of features together was also carried out.

Prognostic Label

An attending radiologist was used to categorize the RIDER test/retestdataset into two broad prognostic groups using quantitative metrics toscore the tumor on a point scale. These parameters included tumor size,differentiation, vascular invasion, margin status (negative vs positiveor close margins, which have all been shown to have prognostic value.Five scalable metrics were used, e.g., lobulated margin, size of thetumor lesion, spiculated margin, plueral wall attachment and texture(e.g. ground glass opacity, GGO), as factors to scale the tumor intohigh risk to moderate risk individuals and used fissure attachment,lymphadenopathy and air bronchogram as a secondary flags to grade thelesions. The score values on the point scale were summarized andstandardized to a scale of 0 to 1. The normalized prognostic score overthe median value was considered high risk (or poor prognosis) versus thesample below, which would mean relatively lower to moderate risk. Twosamples could not be scored reliably using the point scale metric due todisperse lesions. In total four samples were eliminated.

These two categories were then used to find discriminatory markersbetween the poor to better prognosis groups. FIG. 5 illustrates examplesof CT scans for extreme semantic scores, wherein the left panel casesrepresent low scores (e.g., 1 out of 5) and the right panel casesrepresent high scores (e.g., 5 out of 5).

Feature Selection

The reproducibility of radiographic features obtained from CT scans oflung cancer was investigated to establish potential quantitative imagingbiomarkers. Most of the features showed high reproducibility using anautomated image analysis program with segmentation done by a singlereader. Prior work has demonstrated the use of three features:uni-variate, bivariate and volumetric to infer concordance consistencyfor automatic and manually segmented lung lesions, which seems to belimited in describing the complex nature of a tumor. In the currentstudy, the focus has been to describe the tumor with many features usingdifferent categories: size (volume, diameter, border length), shape(shape index, compactness, asymmetry), boundary region (border length,spiculation), relation to the lung field, image intensity based features(mean, standard deviation, average air space, deviation of airspace,energy, entropy, skewness etc.) and transformed texture descriptors(wavelet transform: entropy and energy and laws features). On this setof features, their consistency in repeat scans (test, re-test) wastested and filtered to find independent features. The stable,independent features provide an image feature set that may, for example,be used to predict prognosis.

One requirement for an image feature to be qualified as a responsebiomarker is that the change in its value between pre- and post-therapyscans must be significantly greater than the difference observed in thepresent “Test-Retest” (or “15 minute coffee break”) measurements. In thepresent study, the change of individual features that may be encounteredpost-therapy to be within the entire pre-therapy biological range can beestimated. The ratio of the range to the inter-scan variability is ameasure of “dynamic range. Features showing high dynamic range wereconsidered potentially more informative. The distribution of concordancecoefficients between Test and Re-test, which is skewed toward higher endvalues as one would expect showing high concordance between the Test andRe-Test cases. There is also a larger peak toward zero values,investigating the peaks shows some of the Laws and higher level waveletfeatures have low concordance between Test, Re-Test repeats. It ishypothesized that the reimaging of the patients resulted in some changein texture (perhaps from segmentation differences). These Laws featurescompute energy after filtering in a region, small changes in subregional textures would make these features vary as they capture smalllocalized changes. A similar analogy could be made for wavelet featuresfor higher layer decompositions (or higher layers), where discordancecan be seen.

In prior work, a correlation coefficient threshold of 0.9 was used todistinguish highly correlated features. In the current study, thecoefficient of determination (R²) was used between the features to findthe dependency. It was shown the coefficient of determination in simplelinear regression is equal to the square of the Pearson correlationcoefficient.

Feature Reduction

Feature reduction to select an informative feature set is an activeresearch field, metrics that have been used in the past, include: thecorrelation coefficient, regression methods and classification accuracy.In this study, a representative feature set that will eliminateredundancy in terms of information content is intended to be identified,as complete independence may not be relevant for the study as textureinformation is subjective (and affected by sample issues, scannersetting, protocol followed, etc.). The coefficient of determination (R²_(Bet)) between two features, to form a matrix of all possible pairs toquantify dependency can be used. Features were grouped based on R²_(Bet) between them (over a certain limit); in this subset onerepresentative was picked that had the highest dynamic range. Theprocedure was repeated recursively to cover all the features resultingin a most representative group, done independently for each category.

The test, retest values were averaged before computing R² _(Bet).Different limits were set to combine the features, R² _(Bet) from 0.75to 0.99. For higher thresholds R² _(Bet), less features will be groupedtogether resulting in a larger representative group. Setting the R²_(Bet) to a lower limit will group more features together resulting in asmaller representative feature set (i.e. set of independent features).The combination of reproducibility, plus informative and independentsets of features is critical to obtain a feature set which may containimaging biomarkers. A number of representative features obtained atdifferent thresholds for concordance and dynamic range. As an example,selecting the midlevel threshold CCC_(TreT) & DR≧0.90 yielded 66features. In this subset, the representative features were found with R²_(Bet)≧0.95 resulting in 42 features.

FIG. 6 shows a heatmap of coefficient of determination (R2Bet) betweenthe features, for CCCTreT & DR≧0.90. The features (in the rows) reportedare the ones with CCCTreT & Dynamic Range (DR)≧0.90. The features thathave been picked as representative features with R² value≧0.95 areoutlined with a solid box. The representative features were pickedcategory wise, displayed across all categories. The hierarchicalclustering was stopped arbitrarily at six groups of features and threegroups on the sample side, represented by the multicolor bar. Thedendograms on the top and sides show complete groupings with averagelinkage. The color map of the clustogram ranges from 0 to 1, valuesclose to 1 have a red shade and values close to 0 have a black shade.The groups with red shade mean relatively high feature values.

The image features left out of the representative reduced feature setcould also be useful features. The image features are expected tocapture different aspects of morphology, and texture information. Due tothe consistency in samples chosen and a limited sample population theimage features computed may show a higher level of dependency. Thesamples chosen as primary lung tumor may have a limited amount oftexture or morphological changes.

Results and Discussion

In the obtained regions of interest (ROI), 219 3D features wereextracted, which are described in Tables A1 and A2. The feature nameswere abbreviated to fit them in the table format, for example:‘F78:3D-Laws-20’ would mean, Feature#78 (from the total of 219features), it's a 3D texture feature, computed by the “Laws” kernel oftype 20. The kernel reference can be found in the parenthesis, i.e., ‘E5S5 W5 Layer 1’. All features can be decoded in this way.

Conventional Radiologist Measures: In order to be comparable withprevious work, the concordance correlation confidence limits forsegmentation on three features were compared. As before, highconcordance across test-retest was found. The difference distributionbetween test and retest for the three measures (Length, Area, andVolume) were observed. As the tumor size increased, the differencebetween test & retest was reduced, as observed in previous analyses.

Concordance in the New Features: The 219 extracted features were firstcompared using the Concordance Correlation Coefficient, which is astringent measure of reproducibility. A CCC_(TreT) value≧0.75 indicatesthat the data are of acceptable reproducibility. For the data set,various thresholds were examined with a preference for high stringency.These analyses identified 45, 66 and 93 features that had CCCTreT valuesabove thresholds of 0.95; 0.90 and 0.85, respectively.

Dynamic Range in New Features: At a second level of analysis, thedynamic range was computed as described in Methods, and is a measure ofbiological range for each feature, relative to its reproducibility.Features with a dynamic range≧0.95 have a biological range that is morethan 20-fold greater than the test-retest difference. These analysesidentified e 59, 189 & 219 feature above dynamic range thresholds of0.9, 0.90 & 0.85, respectively. Applying both the filters features wereidentified that passed the threshold set by CCC as well as the Dynamicrange filter. These two filtering procedures will result in a set offeatures that is reproducible and has a large range compared to thevariability between the test and re-test experiments.

Redundancy Reduction: It is known that agnostic features may beinter-dependent. To reduce redundancies, the coefficient ofdetermination (R² _(Bet)) between all possible pairwise combinations offeatures was used to quantify the level of similarity. In this approach,if a feature of interest is linearly predicted by any other feature inthe filtered feature set, then the feature having the largest dynamicrange was chosen as the representative for the group and the other wasremoved. The procedure was repeated to cover the entire subset to formthe reduced set. The threshold level to flag features as linearlydependent is critical and is subject to change with the sample size andthe tumor shape and texture. Using an R² _(Bet) threshold of ≧0.95 toidentify interdependence, there were 42 features that had CCCTreT & DRvalues≧0.90. At a lower setting, of CCC_(TreT) & DR≧0.85, there were 53features.

Practical Application of Repeatable Markers

Quantitative image features have been shown to predict prognosis inprior studies. The objective in this work is to find reproducible,non-redundant and high dynamic range image features that could beprognostic or response markers. Following is a practical example toillustrate the potential utility of these markers, where reproducibilityis a required trait for them to be used as a prognostic predictor.

Prognosis Discrimination: As an application the features that pass theconcordance and dynamic range filters were used to test their ability todiscriminate groups based on their prognostic score, determined asdescribed in methods. The section below describes details on the samplelabel information. Statistical tests (T test and Wilcox rank test) wereapplied to find discriminating image feature between the prognosticgroups. The P-values for the features were obtained and the valuescorrected for multiple testing by computing the false discovery rate(FDR) for the features, performed independently for features in eachcategory. Because they were reproducible, the feature values for thetest and retest were taken as independent observations. The featuresthat had a FDR for both tests of 0.05 were considered to be prognosticdiscriminators.

The sample set was diverse with more large tumors than small ones, andhence most size based features were near the top of the prognosticpredictor list. Size is a well-known prognostic feature for many tumors.In addition, it was also observed that a large number of texturefeatures (Histogram, Laws and wavelets) were prognostic. Notably,texture, size & shape based descriptors showed equal prognostic value.

Optimal Threshold: Linear discriminant analysis was used to find acutoff level for the significant features. Using the prognostic labelsas ground truth sensitivity, specificity and area under the curve (AUC)were computed. Some of the texture features (Runlength, Laws & Wavelets)had to be linearly scaled (by a factor of 1000) before computation toavoid numerical errors.

The ROC (receiver operator characteristics) for two features is shown inFIGS. 7A and 7B, the area under the curve is 90% for the conventionalsize based feature (longest diameter) and 91% for the chosen texture(Run length) feature. Scores were arbitrarily assigned, where below themedian as “good” (in green) and those above the median as “bad” (inred), in a relative sense. Segmented region for representative slicesfrom top to bottom of the tumor, for samples with extreme values oftexture features (also identified to be predictive of prognostic score).The samples in Case 1, FIG. 7A: (a), (b) & (c), (d) had the highest andlowest Average Run length (F54: Avg.Run.L) measure for test/retest. Thefeature value for images in (a),(b) was 22153.85 & 221262.36. While (c),(d) was: 688.56 & 700.96 respectively. The samples in Case 2, FIG. 7B:(a), (b) & (c), (d) had the highest and lowest laws kernel measure (F59:Laws −1: E5 E5 E5 Layer 1) for test/retest. The feature value for imagesin (a), (b) was 0.07404 & 0.07507. While (c), (d) was 0.00864 & 0.00658respectively.

The example in FIGS. 8A and 8B show two extreme sample cases withrepresentative slices for Run length and a Laws kernel feature. FIGS. 8Aand 8B show prognostic texture features relationship to conventionalmeasures (Longest Axis, Short Axis*Longest Axis & Volume). Inparticular, FIG. 8A shows run length features and FIG. 8B shows Lawskernel features. The Laws 1 (E5 E5 E5 Layer 1) is an edge detectingkernel of length 5, applied across all directions (x, y & z) andnormalized based on the size of the tumor. It is expected to have aninverse relation to tumor size and expected to measure tumors edges.FIG. 9 shows the relationship of the prognostic feature to theconventional measures. It shows in the lower range it tracks the sizemeasurement but deviates as the feature value increases. These texturefeatures may capture more information than traditional size basedmeasurements. In particular, FIG. 9 shows three representative slices ofpatient CT scans selected from 39 Adenocarcinoma cases with betterradiological prognostic score (A&B, score of 0.44 & 0.48 respectively)and poor radiological prognostic score (C &D, score of 0.8, 0.92respectively.

Conclusions

The current study demonstrates that the test-retest reproducibility ofmost CT features of primary lung cancer is high when using an imageanalysis program with semi-automated segmentation. Across all patients,the biological ranges for the majority of individual features were veryhigh. The features with lowest inter-scan variance relative to thelargest biological range (i.e. dynamic range, DR) should be explored aspotentially the most informative for use as imaging biomarkers.Additionally, a co-variance matrix of features identified severalredundancies in the feature set that could be combined into a singlevariable. Combining inter-scan variance, biological range andco-variance, it is possible to reduce the total number of features from219 to a most informative set of 42 features identified at a setting ofCCC_(TreT) & DR≧0.9 (R² _(Bet)≧0.95), while a less stringent cutoffthere are 77 features with CCC_(TreT) & DR≧0.85 (R² _(Bet)≧0.99). Thesereproducible and representative features show high ability todiscriminate the tumors based on its prognostic labels. There were 69%of size based features, 62% of histogram features and 29% Run lengthfeatures were able to discriminate tumors with low and high prognosticscores. This allows selection of reproducible, informative andindependent features that are candidate imaging biomarkers to predictprognosis and predict or assess therapy response.

Lexicon Development of Quantitative Computer Tomographic Characteristicsfor Lung Adenocarcinoma and Their Association With Lung Cancer Survival

As described below, 25 CT descriptors have been developed from among 117patients with lung adenocarcinoma and found that, of these, pleuralattachment was most significantly associated with an increased risk ofdeath overall and among patients with adenocarcinomas showing purelepidic growth or with lepidic growth as its predominant component,while lymphadenopathy was significantly associated with an increasedrisk of death among patients with adenocarcinomas without apredominantly lepidic component. An initial set of CT descriptors wasdeveloped to quantitatively assess lung adenocarcinomas in patients(n=117) who underwent resection. Survival analyses were used todetermine the association between each characteristic and overallsurvival. Principle component analysis (PCA) was used to determinecharacteristics that may differentiate histological subtypes.

The twenty-five descriptors were developed using either an ordinal scalefrom 1 to 5 or binary categorical rating. Characteristics significantlyassociated with overall survival included pleural attachment (P<0.001),air bronchogram (P=0.03), and lymphadenopathy (P=0.02). Multivariateanalyses revealed pleural attachment was significantly associated withan increased risk of death overall (Hazard Ratio [HR]=3.42; 95%confidence interval [CI] 1.70-6.86) and among patients withadenocarcinomas showing pure lepidic growth or with lepidic growth asits predominant component (HR=5.98; 95% CI 1.78-20.16), whilelymphadenopathy was significantly associated with an increased risk ofdeath among patients with adenocarcinomas without a predominantlylepidic component (HR=2.90; 95% CI 1.08-7.80). A PCA model showed thattumors with or without predominant lepidic growth were separable andtexture was most important for separating the two subtypes.

As described below, a standardized set of quantitative computertomographic (CT) descriptors of lung adenocarcinoma has been developedand their association with overall survival assessed. This approach hasthe potential to support automated analyses by providing guidance andexpert evaluation of necessary imaging characteristics, and it canultimately be used to develop decision support clinical tools toincrease accuracy and efficiency of radiological diagnosis.

A study included 117 patients diagnosed with histologically confirmedadenocarcinoma of the lung who had surgery for primary lung cancerbetween January 2006 and June 2009. Tumors were further classified intotwo subtypes: (1) Bronchioloalveolar carcinoma (BAC) and adenocarcinomawith a predominant BAC component, and (2) Adenocarcinoma without apredominant BAC component. Since the BAC classification is no longerused, the histopathologic term “lepidic” has been introduced to describethe pattern of adenocarcinomas that possess growth along the surface ofalveolar walls. For this analysis the tumors are classified into twogroups: group 1: adenocarcinoma with evidence of pure lepidic growth orwith lepidic growth as its predominant component; and group 2:adenocarcinoma without a predominant lepidic growth.

All CT scans were performed prior to surgery. Slice thicknesses variedbetween 3 to 6 mm (n=18 with 3 mm, n=24 with 4 mm, n=74 with 5 mm, andn=1 with 6 mm). Ninety-five patients underwent contrast-enhanced CT and22 patients had non-enhanced CT. A clinical radiologist with 7 years ofexperience in chest CT diagnosis read all of the CT images and developedthe descriptors. Terms and descriptions were adapted from the BreastImaging Reporting and Data System (BI-RADS) of the American College ofRadiology. Lung cancer-specific characteristics are included from theFleischner Society and those that have been previously published. Eachdescriptor was rated with either an ordinal scale from 1 to 5 or ratedas a binary (present or not present) categorical variable. Each tumorwas rated by assessing all slices and reporting with a standardizedscoring sheet. Intentionally, the ordinal scale was broadened with thepotential to collapse the resolution as evidence is obtained.

Kaplan-Meir survival curves with the log-rank test were performed usingR version 2.14 (R Project for Statistical Computing,http://www.r-project.org) and multivariable Cox proportional hazardregression was performed using Stata/MP 12.1 (StataCorp LP, CollegeStation, Tex.). Among characteristics that were found to bestatistically significantly associated with overall survival inunivariate analyses, forward selection methods were utilized to modelwhich sets of CT characteristics were associated with overall survival.Also performed was a Classification and Regression Tree (CART) adaptedfor failure time data that used the martingale residuals of a Cox modelto approximate chi-square values for all possible cut-points for thecharacteristics(http://econpapers.repec.org/software/bocbocode/s456776.htm). Principlecomponent analysis (PCA) was used to determine characteristics that maydifferentiate histological subtypes.

Results

CT descriptors and patient cohort: Twenty-five descriptors weredeveloped and used to evaluate lung CT scans from patients with lungadenocarcinomas. The goal was to develop an initial set of descriptorscovering a broad area of characteristics with as much resolution aspossible. As shown in Table 22, these descriptors were classified intothree categories: (1) measures describing the tumor (n=16); (2) measuresdescribing the surrounding tissue (n=5); and (3) measures describingassociated findings (n=4). Among these descriptors, 17 were rated usinga 1-5 ordinal scale and 8 descriptors were binary categorical variables.Fifty-five tumors were classified as group 1, showing pure lepidicgrowth or with lepidic growth as its predominant component; 62 tumorswere classified as group 2, without a predominant lepidic growth and,among these cases, 11 had a small proportion of a lepidic component.

Comparison of lung-tumor descriptors with existing classificationsystems: The set of descriptors was adapted in part from those describedin the BI-RADS lexicon (10, 14). Because a goal was to quantitativelydescribe characteristics within CT lung images with an ordinal scalewhen applicable, the measures do not parallel exactly the relateddescriptions defined in the BI-RADS lexicon. For the shape of the tumor,degrees of sphericity was used to describe the roundness, whereas theBI-RADS uses two shapes (i.e., round or oval). For the margindescription degrees of border definition was used, whereas the BI-RADSuses circumscribed (well-defined or sharply-defined) or indistinct(ill-defined); and fissure or pleural attachment was used to indicatethe obscured margin used by BI-RADS (Table 23). For the density of thetumor, the descriptors were quite different from BI-RADS descriptors dueto the difference between the two organs; focus was on the ground-glassopacity (GGO) and used the lexicon of the Fleischner Society forreference. Calcification is a separate category of BI-RADS and there aremany terms used to describe it because calcification is very importantfor breast imaging to infer malignancy. However, calcification is rarein lung adenocarcinomas, so it was incorporated into the densitydescription. Similar to BI-RADS MRI, the enhancement heterogeneity ofthe tumor was described, but with 5 scales. In addition, some specificterms were added used for lung cancer only (Table 22).

The descriptors were developed specifically for patients withpathologically confirmed lung cancer, which is in contrast with many offeatures defined in the Fleischner Society's glossary. The FleischnerSociety terms are used for thoracic imaging in general, although thedensity descriptors were adapted. Other descriptors that were notincluded in the lexicon of Fleischner Society were adapted from theliterature. In particular, “cavity” and “pseudocavity” used byFleischner Society was combined into “air space” as Matsuki et al didbecause of the difficulty to differentiate them on CT images.

Quantitative CT characteristics and overall survival: CT imaging datawere available on 117 patients (Table 24) but complete survival datawere only available for 105 patients, with 50 patients in group 1 and 55patients in group 2. First was analyzed the association of CT scanningparameters with the overall survival. The slice thickness (P=0.70) wasnot associated with overall survival, although the contrast enhancementstatus (P=0.06) trends towards differences. This result suggests thattechnical factors do not play a large role in the prognostic ability ofthe image features. Based on the distributions in Table 24, theassociation of each of the 25 characteristics was analyzed with overallsurvival. The characteristics statistically significantly associatedwith overall survival (FIG. 1) were pleural attachment (P<0.001), airbronchogram (P=0.03), and lymphadenopathy (P=0.02). Size, lobulation,and thickened adjacent bronchovascular bundle are also associated withoverall survival (P<0.05), yet these results may be influenced by smallsample sizes.

As shown in Table 25, for the multivariable Cox proportional hazardmodels, the main effects for pleural attachment were first determined,air bronchogram, and lymphadenopathy, and then stratified the data bygroup. Pleural attachment (Hazard Ratio [HR]=3.42; 95% confidenceinterval [CI] 1.70-6.86) was statistically significantly associated withan increased risk of death among all patients and among group 1 tumors(HR=5.98; 95% CI 1.78-20.16). For group 2 patients, lymphadenopathy wasassociated with an increased risk of death (HR=2.90; 95% CI 1.08-7.80).A forward stepwise selection approach revealed similar findings. A CARTanalysis was performed for pleural attachment, air bronchogram, andlymphadenopathy, and it was found that patients without pleuralattachment and without lymphadenopathy had significantly improvedsurvival compared to patients with pleural attachment (P<0.001).

Difference between the characteristics of histological subtypes: PCAanalysis of the imaging characteristics demonstrated that the twosubtypes (adenocarcinoma showing pure lepidic growth or with lepidicgrowth as its predominant component and adenocarcinoma without apredominant lepidic growth) were separable. The PCA model explained 14%and 11% of the variation in components one and two, respectively. Theseparation of the two subtypes was mostly along the second PCAcomponent, shown on the y-axis. The PCA loading plot showed that texturewas most important for separating the two subtypes, whereadenocarcinomas showing pure lepidic growth or with lepidic growth asits predominant component tended to have more of a ground-glassappearance, i.e. lower value for the texture characteristic. It is alsonoteworthy that the surrounding tissues and associated findings wereimportant to the PCA model (their loading values were not zero) andadded important information to the tumor characteristics. Interestingly,adenocarcinomas with only a minimal lepidic component also showed someextent of lepidic growth characteristics. Some of the adenocarcinomasbelonging to group 2 that were “misclassified” into group 1 are actuallyadenocarcinomas with a small proportion of a lepidic component.

Thus, of the 25 CT descriptors among 117 patients with lungadenocarcinoma and found that, of these, pleural attachment was mostsignificantly associated with an increased risk of death overall andamong patients with adenocarcinomas showing pure lepidic growth or withlepidic growth as its predominant component, while lymphadenopathy wassignificantly associated with an increased risk of death among patientswith adenocarcinomas without a predominantly lepidic component.

BI-RADS is the most widespread standardized reporting system and wasdeveloped for breast screening. The BI-RADS model was utilized as theguiding principle to develop the descriptors for lung cancer. Thelexicon of the Fleischner Society (11) is well-accepted for thoracicimaging in general, and it was used as a guide for the analysespresented herein. A goal is to develop a lexicon that can supportautomated analysis in the clinical setting by providing guidance andexpert evaluation of important imaging characteristics. In manyinstances documenting the presence of a given characteristic may beinsufficient. For example, previous work used spiculation as onepossible margin rating. In contrast, spiculation was used as a variableunto itself with five degrees.

It is recognized that because the previous term BAC included severalsubcategories of varying histopothologic, radiologic, and prognosticclinical importance, BAC is no longer used according to the newmultidisciplinary classification of lung adenocarcinoma sponsored by theInternational Association for the Study of Lung Cancer, AmericanThoracic Society, and European Respiratory Society in 2011 (15). BAC wasa focus because the 2004 WHO lung cancer classification was used ascriterion when the patients underwent surgery, and adenocarcinomashowing lepidic growth does have unique clinical and radiologicalfeatures. Moreover, BAC is a designation that has been used for over 50years, the problems remaining with the newly developed classificationwere pointed out by some pathologists and whether BAC can reallydisappear from the lexicon has been questioned.

BAC was defined as an adenocarcinoma with pure lepidic growth withoutinvasion of stromal, blood vessels, or pleura. Due to its specificgrowth pattern, BAC usually has unique clinical and radiologicalfeatures. Many investigators have reported a correlation betweenhistopathologic and CT findings in adenocarcinomas. BAC andadenocarcinoma with a predominant BAC component typically show GGO onCT, which reflects the lepidic growth pattern involving alveolar septawith a relative lack of acinar filling. It is generally accepted thatduring the progression from BAC to invasive adenocarcinoma, a GGO noduleincreases in size, after which the solid portion tends to appear,finally, the solid portion increases in extent. The study analyzed thequantitative CT characteristics of adenocarcinomas by using PCA modelingand found adenocarcinoma showing pure lepidic growth or with lepidicgrowth as its predominant component can be separated from adenocarcinomawithout a predominant lepidic growth, and the most importantcharacteristic that differentiated those two subtypes is texture. Thisresult conforms to the progression of BAC. Analyzed further is theadenocarcinomas without a predominant lepidic growth and it wasinteresting that adenocarcinomas with only minimal lepidic componentalso showed some extent of lepidic growth characteristics. These resultssuggest quantitative CT characteristics can be used to predicthistological subtypes of adenocarcinoma based on lepidic component.

Some reports have shown prognostic factors of lung adenocarcinoma fromCT findings (19, 23, 24). Smaller extent of GGO, lack of lobulation orair bronchograms, presence of coarse spiculation, and thickening ofbronchovascular bundles around the tumors were correlated with poorersurvival, which were similar to these results. A relationship betweenextent of GGO with survival was not observed. As this relationship wasreported to be found in small (<3 cm) tumors (19, 23, 24), it should beinvestigated further. In particular, pleural attachment was found to bethe most important characteristics correlated with overall survival,especially for adenocarcinoma showing pure lepidic growth or withlepidic growth as its predominant component. In addition, the prognosticfactors for adenocarcinoma showing pure lepidic growth or with lepidicgrowth as its predominant component were found to be different fromthose for adenocarcinoma without a predominant lepidic growth. Thissuggests that the different histological subtypes of adenocarcinomabased on lepidic component should be analyzed separately whenquantitative CT characteristics are assessed for their association withlung cancer outcomes.

There is increased interest and awareness in quantitative imaging,particularly in the context of automated CT image analysis. This lexiconprovides a standard against which quantitative imaging features may bedesigned and compared. Further, a goal was to enumerate as broad adescriptor set as possible. This would provide the opportunity foranalytical techniques to be designed to detect features orcharacteristics not detectable by the human eyes. Such features caneasily be compared against the lexicon of the present disclosure todemonstrate their uniqueness with respect to radiological observations.

In conclusion, the initial results of the study show that quantitativeCT characteristics were associated with overall survival in a cohort oflung adenocarcinoma patients. Specifically, pleural attachment wasassociated with an increased risk of death, especially amongadenocarcinomas showing pure lepidic growth or with lepidic growth asits predominant component. The retrievable data elements in thequantitative CT characteristics can be used for data mining anddeveloping automated objective features, which would benefit therapyplanning and ultimately improve patient care.

TABLE 22 Lexicon of CT characteristics for lung adenocarcinomaCharacteristic Definition Scoring Scoring definition Tumor SpaceLocalization Location Lobe location of the tumor 1, 2, 3, 4, 5 1-rightupper lobe; 2-right middle lobe; 3-right lower lobe; 4-left upper lobe;5-left lower lobe Distribution Central location: tumor located in thesegmental or more proximal bronchi Peripheral location: tumor 0, 10-central; 1- located in the subsegmental peripheral bronchi or moredistal airway Fissure tumor attaches to the fissure, 0, 1 0-no; 1-yesattachment tumor's margin is obscured by the fissure Pleural tumorattaches to the pleura 0, 1 0-no; 1-yes attachment other than fissure,tumor's margin is obscured by the pleura the greatest dimension in lung1, 2, 3, 4, 5 1-≦2 cm; 2->2-3 cm; window 3->3-5 cm; 4- >5-7 cm; 5->7 cmSize Shape Sphericity roundness 1, 2, 3, 4, 5 1-round; 2, 3, 4, 5-elongated with increasing degrees Lobulation contours with undulations1, 2, 3, 4, 5 1-none; 2, 3, 4, 5- lobulated with increasing degreesConcavity concave cuts 1, 2, 3, 4, 5 1-none; 2, 3, 4, 5- concaved withincreasing degrees Irregularity complex shape 1, 2, 3, 4, 5 1-smooth; 2,3, 4, 5- irregular with increasing degrees Margin Border well orill-defined border 1, 2, 3, 4, 5 1-well defined; definition 2, 3, 4,5-poorly defined with increasing degrees Spiculation lines radiatingfrom the margins 1, 2, 3, 4, 5 1-none; spiculated of the tumor withincreasing degrees Density Texture solid or ground-glass opacity 1, 2,3, 4, 5 1-Nonsolid (pure ground-glass opacity); 2-Partly solid, smallextent of solid component; 3- Partly solid, large extent of solidcomponent; 4- solid, with relatively low density; 5-solid Air spaceincluding cavity and 1, 2, 3, 4, 5 1-None; 2, 3, 4, 5- pseudocavity dueto the from small to large difficulty to differentiate on CT extent oflucencies images Air bronchogram tubelike or branched air 1, 2, 3, 4, 51-None; 2, 3, 4, 5- structure within the tumor from small to largeextent of air bronchogram Enhancement heterogeneity of tumor on 1, 2, 3,4, 5 1-homogeneous; heterogeneity contrast-enhanced images 2, 3, 4,5-from mildly to highly heterogeneous Calcification any patterns ofcalcification in 0, 1 0-no; 1-yes the tumor Surrounding tissues Pleuralretraction retraction of the pleura towards 1, 2, 3, 4, 5 1-None; 2, 3,4, 5- the tumor from slight to obvious pleural retraction Vascularconvergence convergence of vessels to the 1, 2, 3, 4, 5 1-None; 2, 3, 4,5- tumor, only applied to the from slight to peripheral tumors obviousvascular convergence Thickened adjacent widening of adjacent 1, 2, 3, 4,5 1-None; 2, 3, 4, 5- bronchovascular bronchovascular bundle fromslightly to bundle obviously thickened Emphysema peripheral emphysemacaused 1, 2, 3, 4, 5 1-none; 2, 3, 4, 5- periphery by the tumor orpreexisting from mild to emphysema severe emphysema Fibrosis peripheryperipheral fibrosis caused by the 1, 2, 3, 4, 5 1-none; 2, 3, 4, 5-tumor or preexisting fibrosis from mild to severe fibrosis Associatedfindings Nodules in primary any nodules suspected 0, 1 0-no; 1-yes tumorlobe malignant or indeterminate Nodules in nontumor any nodulessuspected 0, 1 0-no; 1-yes lobes malignant or indeterminateLymphadenopathy thoracic lymph nodes with short 0, 1 0-no; 1-yes axisgreater than 1 cm Vascular involvement vessels narrowed, occluded or 0,1 0-no; 1-yes encased by the tumor, only applied to the contrast-enhanced images

TABLE 23 Differences between our descriptors and BI-RADS (Fourth editionfor Mammography) for shape and margin BI-RADS Descriptors Scale ShapeRound Sphericity 1 Oval Sphericity 2 to 5 Lobular Lobulation 3 to 5Irregular Irregularity 2 to 5 Concavity 1 to 5 Margin Circumscribed(Well-Defined or Border definition 1 Sharply-Defined) MicrolobulatedLobulation* 2 Obscured Fissure or pleural attachment^(†) Indistinct(III-Defined) Border definition 2 to 5 Spiculated Spiculation 2 to 5 *Inshape group ^(†)In space localization group

TABLE 24 Distribution of the CT imaging characteristics among 117 lungcancer patients CT characteristic N (%) Location 1 35 (29.9) 2 9 (7.7) 320 (17.1) 4 37 (31.6) 5 16 (13.7) Distribution 0 9 (7.7) 1 108 (92.3)Fissure attachment 0 81 (69.2) 1 36 (30.8) Pleural attachment 0 82(70.1) 1 35 (29.9) Size 1 29 (24.8) 2 46 (39.3) 3 35 (29.9) 4 7 (6.0)Sphericity 1 7 (6.0) 2 18 (15.4) 3 65 (55.6) 4 24 (20.5) 5 3 (2.6)Lobulation 1 3 (2.6) 2 34 (29.1) 3 48 (41.0) 4 28 (23.9) 5 4 (3.4)Concavity 1 6 (5.1) 2 37 (31.6) 3 38 (32.5) 4 28 (23.9) 5 8 (6.8)Irregularity 2 22 (18.8) 3 44 (37.6) 4 34 (29.1) 5 17 (14.5)

TABLE 25 CT characteristic N (%) Border definition 1 2 (1.7) 2 50 (42.7)3 40 (34.2) 4 17 (14.5) 5 8 (6.8) Spiculation 1 6 (5.1) 2 46 (39.3) 3 33(28.2) 4 26 (22.2) 5 6 (5.1) Texture 1 3 (2.6) 2 18 (15.4) 3 35 (29.9) 430 (25.6) 5 31 (26.5) Air space 1 65 (55.6) 2 30 (25.6) 3 16 (13.7) 4 5(4.3) 5 1 (0.9) Air bronchogram 1 75 (64.1) 2 26 (22.2) 3 8 (6.8) 4 6(5.1) 5 2 (1.7) Enhancement heterogeneity 1 2 (1.7) 2 10 (8.6) 3 34(29.1) 4 28 (23.9) 5 18 (15.4) N/A 25 (21.4) Calcification 0 111 (94.9)1 6 (5.1) Pleural retraction 1 12 (10.3) 2 34 (29.1) 3 42 (35.9) 4 26(22.2) 5 3 (2.6) Vascular convergence 1 23 (19.7) 2 35 (29.9) 3 22(18.8) 4 24 (20.5) 5 6 (5.1) NA 7 (6.0) Thickened adjacentbronchovascular bundle 1 83 (70.9) 2 15 (12.8) 3 17 (14.5) 4 1 (0.9) 5 1(0.9) Emphysema periphery 1 52 (44.4) 2 25 (21.4) 3 28 (23.9) 4 9 (7.7)5 3 (2.6) Fibrosis periphery 1 7 (6.0) 2 27 (23.1) 3 50 (42.7) 4 26(22.2) 5 7 (6.0) Nodules in primary tumor lobe 0 60 (51.3) 1 57 (48.7)Nodules in non-tumor lobes 0 45 (38.5) 1 72 (61.5) Lymphadenopathy 0 83(70.9) 1 34 (29.1) Vascular involvement 0 33 (28.2) 1 61 (52.1) N/A 23(19.7)

TABLE 26 Hazard ratios for the association between the CTcharacteristics and overall survival Overall Group 1³ Group 2³ mHR (95%CI)¹ mHR (95% CI)¹ mHR (95% CI)¹ Main effects Pleural 3.42 (1.70-6.86)5.98 (1.78-20.16) 2.22 (0.85-5.83) attachment Air 1.48 (0.76-2.91) 1.11(0.33-3.73) 1.47 (0.60-3.61) bronchogram Lymphad- 1.88 (0.91-3.87) 0.44(0.50-3.84) 2.90 (1.08-7.80) enopathy Final Model² Pleural 3.56(1.74-7.29) 5.98 (1.78-20.16) NI attachment Air NI NI NI bronchogramLymphad- 2.00 (0.94-4.25) NI 2.90 (1.08-7.80) enopathy Bolded valuesindicate a statistically significant result ¹Adjusted for age, race,gender, smoking status, histological subtype, and stage, whereappropriate ²The finals were derived from forward stepwise regressionmodeling with a 0.1 significance level for inclusion into the model.Adjustment factors were forced into the forward stepwise selectionmodel. The features that were not included in the final model weredesignated as ‘NI’ ³Group 1 are adenocarcinoma tumors showing purelepidic growth or with lepidic growth as its predominant component;group 2 are adenocarcinoma tumors without a predominantly lepidiccomponent

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described above.Rather, the specific features and acts described above are disclosed asexample forms of implementing the claims.

1. A method for diagnosing tumors in a subject by performing aquantitative analysis of a radiological image, comprising: identifying aregion of interest (ROI) in the radiological image; segmenting the ROIfrom the radiological image; identifying a tumor object in the segmentedROI; segmenting the tumor object from the segmented ROI; extracting aplurality of quantitative features describing the segmented tumorobject; and classifying the tumor object based on the extractedquantitative features, wherein the quantitative features include one ormore texture-based features.
 2. The method of claim 1, whereinclassifying the tumor object based on the extracted quantitativefeatures further comprises predicting whether the tumor object is amalignant or benign tumor.
 3. The method of claim 1, wherein classifyingthe tumor object based on the extracted quantitative features furthercomprises using a decision tree algorithm, a nearest neighbor algorithmor a support vector machine.
 4. The method of claim 1, wherein each ofthe texture-based features describes a spatial arrangement of imageintensities within the tumor object.
 5. The method of claim 1, whereinthe texture-based features include at least one of a run-length texturefeature, a co-occurrence texture feature, a Laws texture feature, awavelet texture feature or a histogram texture feature.
 6. The method ofclaim 1, wherein the quantitative features include one or moreshape-based features.
 7. The method of claim 6, wherein each of theshape-based features describes a location, a geometric shape, a volume,a surface area, a surface-area-to-volume ratio or a compactness of thetumor object.
 8. The method of claim 1, wherein a total number ofquantitative features is greater than approximately
 200. 9. The methodof claim 8, wherein a number of texture-based features is greater thanapproximately
 150. 10. The method of claim 1, wherein the ROI is a lungfield.
 11. The method of claim 1, wherein the radiological image is alow-dose computed tomography (CT) image.
 12. The method of claim 1,further comprising: reducing the extracted quantitative features to asubset of extracted quantitative features; and classifying the tumorobject based on the subset of extracted quantitative features.
 13. Themethod of claim 12, wherein the subset of extracted quantitativefeatures includes one or more quantitative features that are predictiveof the classification of the tumor object.
 14. The method of claim 13,further comprising determining the one or more quantitative featuresthat are predictive of the classification of the tumor object using atleast one of a Recursive Elimination of Features (Relief-F) algorithm, aCorrelation-Based Feature Subset Selection for Machine Learning (CFS)algorithm or a Relief-F with Correlation Detection algorithm.
 15. Themethod of claim 12, wherein the subset of extracted quantitativefeatures includes one or more non-redundant quantitative features havingadequate reproducibility and dynamic range.
 16. The method of claim 15,wherein reducing the extracted quantitative features to a subset ofextracted quantitative features further comprises eliminating one ormore of the extracted quantitative features having a reproducibilitymetric between a baseline radiological image and a subsequentradiological image less than a predetermined reproducibility value, thesubsequent radiological image being captured a fixed period of timeafter the baseline radiological image was captured.
 17. The method ofclaim 16, wherein the reproducibility metric is a concordancecorrelation coefficient.
 18. The method of claim 1, wherein thepredetermined reproducibility value is less than 0.90.
 19. The method ofclaim 1, wherein reducing the extracted quantitative features to asubset of extracted quantitative features further comprises eliminatingone or more of the extracted quantitative features having a dynamicrange less than a predetermined dynamic range value.
 20. The method ofclaim 1, wherein reducing the extracted quantitative features to asubset of extracted quantitative features further comprises eliminatingone or more of the extracted quantitative features that are redundantquantitative features.
 21. The method of claim 20, further comprisingcalculating a coefficient of determination (R² _(Bet)) between at leasttwo quantitative features, wherein the R² _(Bet) of each of theredundant quantitative features is less than a predetermined redundancyvalue.
 22. The method of claim 21, wherein predetermined redundancyvalue is 0.95.